Multi-Agent Query System for Industrial Repair Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern query-answering agents, such as those based on Large Language Models (LLMs), can produce incorrect responses or provide incorrect justifications, leading to the need for domain experts to evaluate accuracy in high-stakes situations.
Innovation Solution
A query-answering system that utilizes multiple sets of query-answering agents for answer formulation, incorporating intermediate automatic response validation and consensus-based answer formulation to increase accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single query-answering agent is used, then the system is simple and fast, but the accuracy and reliability of responses deteriorate
Solution Approach 1:
The system divides the query-answering function into multiple independent agents, each responsible for generating a separate response. This segmentation allows each agent to operate independently while their responses are aggregated later, improving reliability without requiring each individual agent to be perfectly accurate
Solution Approach 2:
Multiple agent responses are merged through an aggregation function that combines their outputs into a final answer. This merging process leverages the collective intelligence of multiple agents, where the aggregate response is more reliable than any single agent's response, effectively resolving the contradiction between simplicity and reliability
2Reliability
If multiple query-answering agents are used for answer formulation, then the accuracy and reliability of responses improve, but the system complexity and processing time increase
Solution Approach 1:
The system performs preliminary actions by having each agent generate and evaluate responses in parallel before final aggregation. The evaluation function assesses each agent's response quality upfront, allowing the system to identify and prioritize high-quality responses without waiting for complete processing of all agents sequentially
Solution Approach 2:
The system uses a minimum threshold number of agents (e.g., at least 3) rather than requiring all possible agents to process every query. This partial action approach provides sufficient accuracy improvement while avoiding the excessive time cost of involving too many agents, resolving the contradiction between accuracy improvement and time loss
3Reliability
If domain experts manually evaluate each response, then the accuracy can be verified, but the productivity and efficiency of the system deteriorate
Solution Approach 1:
The system implements self-service through an automated evaluation function that independently assesses each agent's response quality without human intervention. The evaluation function uses predefined criteria and scoring mechanisms to automatically determine response quality, allowing the system to self-verify accuracy while maintaining high productivity
Solution Approach 2:
The evaluation function provides feedback to the aggregation process by assigning quality scores to each agent's response. This feedback mechanism allows the system to automatically adjust which responses are prioritized in the final aggregation, ensuring high accuracy while maintaining efficiency through automated decision-making rather than manual expert review
Data Source
AI summary
A system and method comprising receiving a query related to an industrial unit, generating a consultation prompt based on the query; retrieving, using a consultation agent, a response; generating, based on the response, an evaluation prompt; retrieving, using an evaluation agent, a response evaluation; responsive to determining that response evaluation is positive, generating a summarization prompt based on the response; retrieving, using a summarization agent, a response summary, and storing the response summary in a response set used to generate an answer comprising a repair plan, the answer to be transmitted to a scheduling system for scheduling repairs to the industrial unit. The response set further comprises a predetermined minimum number of response summaries from multiple summarization agents and corresponding to responses from multiple consultation agents, the responses evaluated by multiple evaluation agents. The answer to the query is further generated, based on the response summaries, using an aggregation agent.