Multi-LLM Output Extraction with Model Ranking and Refinement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) often produce incorrect, biased, or misleading information due to training data limitations and inherent stochastic nature, and there is a challenge in effectively mapping queries to the most suitable model and consolidating their outputs for coherent and accurate responses.
Innovation Solution
A system and method for training a model to rank LLMs based on labeled data, selecting optimal LLMs, sending prompts, and refining responses using reinforcement learning and self-training to integrate outputs from multiple LLMs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple LLMs are used to handle diverse applications, then the capability and versatility of the system is improved, but the complexity of mapping queries to suitable models and consolidating outputs increases
Solution Approach 1:
A meta-model is introduced as an intermediary component that receives user queries, selects appropriate LLMs based on task requirements, and consolidates their outputs into unified responses. This mediator layer resolves the complexity of directly managing multiple LLMs by abstracting their differences into a standardized interaction interface.
Solution Approach 2:
The system segments the LLM selection and output consolidation tasks into separate functional modules within the meta-model. The selection module identifies suitable models based on query characteristics, while the consolidation module integrates their responses, allowing independent optimization of each function.
2Adaptability or versatility
If LLMs are used for critical applications, then the functionality and capability is improved, but the reliability and accuracy of outputs deteriorates due to training data limitations and stochastic nature
Solution Approach 1:
The system merges outputs from multiple LLMs through the meta-model's consolidation mechanism. By aggregating responses from different models and selecting the most accurate information, the system compensates for individual model errors and stochastic variations, thereby improving overall reliability.
Solution Approach 2:
The meta-model incorporates feedback mechanisms that evaluate LLM outputs against ground truth or consistency criteria. This feedback loop enables the system to identify and correct inaccurate responses, continuously improving the reliability of outputs in critical applications.
Data Source
AI summary
A system and method for extracting data from large language model (LLM) outputs, including: training a model using labeled data items to assign rankings to LLMs; selecting, by the trained model, one or more of the LLMs based on the rankings; sending an LLM prompt to selected models; and outputting, by the model, a refined response to the prompt based on responses to the prompt by the LLMs. Some LLM prompts according to some embodiments may include different sets input parameters of different types—such as, e.g., a set of block parameters and a set of editorial parameters. In some embodiments, a model or LLM may be updated or retrained using a reinforcement learning approach and based output items or refined responses generated by that model or LLM-which may for example be scored or ranked and used in combination with reward or cost functions to update model parameters.