RAG Context Recommendation for Accurate Multi-LLM Query Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) often generate factually unsupported content or responses that are not responsive to queries, and current approaches to addressing this issue are not effective, especially in large enterprises with multiple generative AI programs and retrieval augmented generation (RAG) architectures that lack consistency and efficiency in selecting and managing context stores.
Innovation Solution
A context recommendation platform utilizing a deep neural network-based classification algorithm to predict the right context store for queries, leveraging historical data and user feedback to dynamically update the quality of context stores, and interfacing with multiple RAG architectures to ensure accurate and efficient responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple LLMs and RAG architectures are deployed in large enterprises, then the system can handle diverse queries and domains, but the complexity of selecting and managing context stores increases significantly
Solution Approach 1:
A recommendation system is introduced as an intermediary layer between the query and the LLM/RAG architecture. This recommendation system uses historical data and machine learning to predict the optimal context store and LLM combination, automatically managing the complexity of selection and reducing the burden on enterprises to manually configure and manage multiple architectures.
2Ease of operation
If LLMs are used to generate responses, then the system can provide natural language answers, but the models may generate factually unsupported content or hallucinations
Solution Approach 1:
The system incorporates feedback mechanisms that use historical data about which context stores and LLM combinations produced the most accurate and reliable responses. This feedback is used to train recommendation models to predict better combinations for future queries, gradually improving factual accuracy while maintaining natural language generation capabilities.
Solution Approach 2:
The recommendation system performs preliminary selection of the optimal context store and LLM combination before the actual query processing. By predicting the best combination in advance based on historical performance data, the system ensures that factually accurate and reliable models are used from the outset, reducing hallucinations before they occur.
3Measurement precision
If context stores are dynamically selected based on query analysis, then response accuracy improves, but processing time increases due to analysis and prediction steps
Solution Approach 1:
The system performs context store and LLM combination selection in advance, before the actual query processing begins. By predicting the optimal combination based on query analysis and historical data, the system prepares the necessary context and model configurations ahead of time, reducing the processing time during actual query response generation while maintaining high accuracy.
Data Source
AI summary
A method comprises receiving a large language model request, analyzing the large language model request using one or more machine learning algorithms, and predicting, based at least in part on the analyzing: (i) a large language model of a plurality of large language models to process and to respond to the large language model request; and (ii) at least one database from which data is to be used to generate a prompt for the large language model. The method further comprises interfacing with the large language model and the at least one database to enable the large language model to process and to respond to the large language model request.