Multi-Agent Response Synthesis for Coherent LLM Answers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated assistants face challenges in combining responses from multiple large language models (LLMs) into a coherent answer suitable for users, as different LLMs excel in different query types, leading to inconsistent or inaccurate responses.
Innovation Solution
A method and system that decomposes user queries into sub-queries based on context, routes each to a corresponding agent with a configured LLM, generates responses, and combines them using similarity scores to ensure context consistency and accuracy, presenting a unified answer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple LLMs are used to answer different types of user questions, then response quality for specific query types is improved, but combining responses into a single coherent answer becomes difficult
Solution Approach 1:
The system segments a user query into multiple sub-queries, each assigned to a specialized LLM agent based on its expertise. This allows each agent to handle only the portion of the query it is best suited for, improving response quality while managing complexity through division of labor.
Solution Approach 2:
A router component acts as an intermediary that receives the original query, decomposes it into sub-queries, selects appropriate LLM agents, and integrates their responses into a final coherent answer. This mediator manages the complexity of coordinating multiple specialized agents.
2Adaptability or versatility
If responses from multiple LLMs are combined, then coverage of different query types is improved, but response consistency and accuracy deteriorate
Solution Approach 1:
Each LLM agent is trained with specialized knowledge and context relevant to its specific query type (e.g., tax preparation, account maintenance, fraud prevention). This local expertise ensures that each agent produces consistent and accurate responses within its domain while the system as a whole handles diverse query types.
3Measurement precision
If similarity scoring is used to verify response-context alignment, then response accuracy is improved, but processing time increases
Solution Approach 1:
The system performs similarity scoring selectively - comparing each response only with its corresponding sub-query context and the original user query. This partial verification approach ensures adequate accuracy checking without excessively processing every possible combination of responses and contexts.
Data Source
AI summary
Aspects of the present disclosure relate to an online resource that can initiate a conversation between a user and an automated assistant. The online resource receives from the user a query including a plurality of sub-queries, determines a context for each of the sub-queries, identifies a plurality of queries from the user during the conversation between the user and the automated assistant, and obtains responses to the sub-queries from a plurality of selected agents. The online resource determines a similarity score for each response by comparing the response with at least the context for the corresponding sub-query, summarizes the responses based at least in part on the similarity scores, and generates an answer to the query by combining the response summaries based at least in part on an alignment between the response summaries and their corresponding sub-queries.