LLM Query Routing With Specialized Agents for Multi-Part Requests
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for processing multi-part user queries or queries using generative AI models fail to efficiently address complex human language queries or queries involving complex human language queries or queries involving complex human language queries or queries involving complex human language queries or queries.
Innovation Solution
A method for decomposing user queries into sub-queries and selecting appropriate agents associated with large language models (LLMs) trained for specific contexts to perform functions such as generating documents, completing forms, or sending emails, ensuring accurate and timely responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single generic LLM is used to handle all user queries, then the system is simple to manage, but the accuracy and effectiveness for complex multi-part queries deteriorates
Solution Approach 1:
The patent segments the query handling system into multiple specialized LLM agents, each trained for specific contexts or query types. The router decomposes complex user queries into sub-queries and directs them to appropriate specialized agents, improving accuracy while maintaining manageable system complexity through modular architecture.
Solution Approach 2:
The patent creates a universal query handling system where multiple LLM agents can handle different types of queries through a common routing mechanism. The router provides multi-functionality by directing various query types to appropriate specialized agents, allowing the system to handle diverse queries effectively without requiring separate dedicated systems for each query type.
2Reliability
If multiple specialized LLM agents are used for different query types, then the query handling accuracy improves, but the system complexity increases
Solution Approach 1:
The patent introduces a router as an intermediary component that manages the complexity of multiple specialized LLM agents. The router receives user queries, decomposes them into sub-queries, and directs each sub-query to the most appropriate specialized agent based on context analysis, thereby shielding users from system complexity while maintaining high accuracy.
Solution Approach 2:
The patent implements dynamic query routing where the system adapts its agent selection based on the specific query content and context. The router dynamically determines which specialized agents to invoke for each query, allowing the system to optimize its complexity usage by only activating necessary agents rather than maintaining all agents in constant operation.
3Productivity
If complex queries are processed without decomposition, then the processing speed is faster, but the handling accuracy for multi-part queries deteriorates
Solution Approach 1:
The patent segments complex user queries into smaller sub-queries that can be handled by specialized LLM agents. This decomposition improves accuracy for multi-part queries by allowing each sub-query to be processed by the most appropriate specialized agent, while the overall process remains efficient through parallel processing of sub-queries.
Solution Approach 2:
The patent performs preliminary query decomposition and routing before actual query processing. The router analyzes the user query, identifies appropriate specialized agents, and prepares sub-queries in advance, which streamlines the subsequent processing steps and maintains overall processing efficiency while improving accuracy.
Data Source
AI summary
Implementations of the subject matter described in this disclosure may be used to process queries received from a user of an online resource and route the received queries to various agents that are determined to be the most suitable for performing one or more functions in response the user queries. For each of one or more received queries, an example method may determine a function corresponding to the one or more queries, select, for each function, at least one agent of a plurality of agents based on the one or more queries, send the one or more queries to a respective agent of the selected one or more agents, and receive, from a responding agent of the selected at least one agent, at least one of a document, a message, or a link representing a result of performing the function.