Language Model Routing via Intermediary Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing dialog systems face challenges in accurately selecting the most suitable language model for user utterances, especially when dealing with ambiguous or rule-noncompliant inputs, which can lead to suboptimal dialog experiences and increased costs.
Innovation Solution
An information processing device that includes a control unit capable of generating a prompt text with a list of specialized language models, inputting this text into a second language model, and identifying the most suitable first language model based on the response, thereby transferring the user's utterance to the appropriate model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single large-scale language model is used to handle all types of user inputs, then the system can process any query, but the cost increases and accuracy decreases for domain-specific queries
Solution Approach 1:
The patent divides the language model processing into multiple specialized segments (first language models for specific domains and a second language model for general queries). Each language model is trained for a particular domain, allowing the system to achieve high accuracy for domain-specific queries while avoiding the need to use a single large-scale model for all inputs, thereby reducing computational costs.
Solution Approach 2:
The patent applies local quality by assigning different language models with different expertise to different types of queries. Instead of using a uniform model for all inputs, the system selects the appropriate specialized model based on the query domain, ensuring that each query is processed by the model with the most relevant local expertise.
2Reliability
If multiple specialized language models are used to improve accuracy for different domains, then dialog accuracy improves, but the complexity of model selection increases
Solution Approach 1:
The patent introduces a second language model as an intermediary that receives the user input and determines which first language model should process it. This intermediary model simplifies the selection process by automatically analyzing the input and routing it to the appropriate specialized model, reducing the complexity that would otherwise be required for manual or complex automated model selection.
Solution Approach 2:
The system implements self-service by enabling the second language model to autonomously determine the appropriate first language model based on the input characteristics. The model selection process is self-contained and automatic, requiring no external intervention or complex decision-making infrastructure beyond the second language model's inherent capabilities.
3Ease of operation
If a rule-based approach is used to select language models, then the system is simple to operate, but it fails to handle ambiguous or rule-noncompliant inputs effectively
Solution Approach 1:
The patent replaces the mechanical rule-based selection system with a language model-based selection system. Instead of relying on predefined rules and keywords to determine model selection, the second language model uses its linguistic understanding capabilities to analyze ambiguous inputs and make intelligent routing decisions, significantly improving adaptability while maintaining operational simplicity.
Data Source
AI summary
A prompt text including a list of a plurality of first language models each specialized for intention understanding in a plurality of predetermined domains and a content of the utterance is generated, the prompt text is input to a second language model, the first language model having a domain corresponding to the content of the utterance is identified based on the content of the response obtained from the second language model, and the content of the utterance is transferred to the identified first language model.