RAG Document Ranking With Preference Optimization for Answer Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) deployed in enterprise environments generate outdated or incorrect answers due to outdated training data, and current retrieval-augmented generation (RAG) systems face challenges in assessing reference document quality, context size limitations, and biased document ordering, leading to suboptimal answer generation.
Innovation Solution
Implement a document selection LLM with a document ranking and preference optimization layer to optimize reference document selection and ordering, using a retrieval optimization engine to enhance the relevance and accuracy of answers generated by field LLMs through a system comprising a document selection LLM, ranking LLM, and retrieval optimization engine.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If retrieval-augmented generation systems use external data to augment user queries, then the accuracy of answers is improved, but the complexity of the system increases due to separate document retrieval systems
Solution Approach 1:
The patent merges the document retrieval system and the large language model into a unified architecture where the LLM directly processes both user queries and retrieved documents. This integration eliminates the need for separate retrieval systems while maintaining the ability to provide accurate, context-aware answers through the combined strengths of retrieval and generation capabilities.
Solution Approach 2:
The large language model is designed to perform multiple functions: it acts as both the query processing system and the document retrieval system. The model can retrieve relevant documents, process user queries, and generate accurate answers all within a single unified system, reducing overall system complexity while maintaining high reliability.
2Speed
If standard retrieval systems retrieve documents based on simple relevance scoring, then the speed of retrieval is improved, but the quality of document selection deteriorates due to biased ordering and inability to assess document quality
Solution Approach 1:
The patent implements a feedback mechanism where the large language model evaluates the quality and relevance of retrieved documents and adjusts the retrieval process accordingly. The model provides feedback on document quality, allowing for iterative refinement of document selection to ensure high precision while maintaining efficient retrieval speeds through optimized processing.
Solution Approach 2:
The system changes the parameters of document retrieval by using the large language model to dynamically adjust retrieval criteria based on query context and document characteristics. This allows the system to optimize both speed and quality by adapting retrieval parameters in real-time rather than relying on fixed scoring mechanisms.
3Quantity of substance
If the document list includes all retrieved reference documents, then the completeness of information is improved, but the processing time increases due to context size limitations
Solution Approach 1:
The patent extracts only the most relevant and high-quality documents from the complete retrieved document list and presents these to the large language model for processing. This extraction mechanism ensures that the model processes a manageable subset of information that maintains completeness of key insights while significantly reducing processing time by eliminating redundant or low-quality documents.
Solution Approach 2:
The system applies partial action by selecting a representative subset of documents rather than processing all retrieved documents. This approach maintains sufficient information completeness for accurate answer generation while reducing processing time through selective document processing based on relevance and quality assessments.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method includes applying a document ranking layer of a document selection large language model (LLM) to a document list including multiple reference documents to obtain a ranked document list. The method further includes selecting a subset of reference documents from the ranked document list and processing a user prompt and the document subset by a field LLM to generate an answer. The method further includes ranking the answer with an answer score by a ranking LLM. The method further includes ranking the document subset by the ranking LLM to obtain a ranked document subset. The method further includes calculating a loss function of a preference optimization layer of the document selection LLM based on the answer score and updating at least one training parameter of a foundation model of the document selection LLM based on the loss function of the preference optimization layer.