LLM Selection Networks for Consistent Query Response Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) and generative AI systems exhibit non-deterministic behavior due to varying training data ingestion and algorithms, leading to inconsistent responses for the same queries.
Innovation Solution
A computing platform trains an LLM selection model using historical information and model network data to select the most accurate LLM for a given query, employing a social network of AI systems to refine responses through dynamic feedback loops.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single LLM is used to provide responses, then the system is simple and fast, but the response accuracy and consistency deteriorate due to non-deterministic behavior
Solution Approach 1:
The system segments the LLM evaluation process into distinct components: an LLM selection model that evaluates multiple LLMs, a confidence level determination mechanism, and a selection decision module. This segmentation allows each component to specialize in a specific function, improving overall response accuracy while maintaining manageable system complexity through modular architecture.
Solution Approach 2:
The LLM selection model acts as an intermediary between the user query and the multiple LLMs. It evaluates candidate LLMs, determines confidence levels, and selects the most appropriate LLM for each query. This intermediary layer isolates the complexity of managing multiple LLMs from the end user while ensuring consistent and accurate responses.
2Reliability
If multiple LLMs are evaluated and selected based on confidence levels, then response accuracy improves, but the processing time and computational resources increase
Solution Approach 1:
The LLM selection model is pre-trained on historical information and model network information, establishing a knowledge graph that captures expertise patterns of different LLMs. This preliminary action allows the system to quickly evaluate candidate LLMs during runtime by querying the pre-built knowledge graph, rather than performing exhaustive evaluations from scratch, thus reducing processing time while maintaining high selection accuracy.
Solution Approach 2:
The system dynamically adjusts the confidence threshold parameter based on query characteristics and historical performance data. By changing this parameter, the system can balance between evaluating more LLMs (higher accuracy) and limiting evaluations (faster processing). This parameter adjustment allows flexible optimization of the trade-off between response accuracy and processing time for different operational contexts.
3Measurement precision
If historical information and model network information are used to train the selection model, then selection accuracy improves, but the training complexity and data requirements increase
Solution Approach 1:
The knowledge graph structure serves multiple functions: it stores LLM expertise information, captures relationships between LLMs and query types, and provides a framework for both training and inference. This multi-functionality allows the system to leverage the same data structure for both training the selection model and performing rapid evaluations, reducing training complexity while improving selection accuracy through comprehensive use of historical and network information.
Data Source
AI summary
A computing platform may train, for a first LLM and using historical information for a plurality of LLMs and model network information, an LLM selection model to select one of the plurality of LLMs for providing a response to an input query. The computing platform may input, into the first LLM, an LLM prompt, which may cause the first LLM to generate an LLM output by: 1) comparing a first confidence level that the first output will be accurate to a confidence threshold, 2) based on identifying that the first confidence level meets or exceeds the confidence threshold, generating, using the first LLM, the LLM output, and 3) based on identifying that the first confidence level fails to meet the confidence threshold: identifying, using the LLM selection model, an alternative LLM of the plurality of LLMs, and input the LLM prompt into the alternative LLM to produce the LLM output.