Knowledge Re-Ranking for Accurate LLM Context Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing digital assistants based on large language models (LLMs) face limitations in understanding natural language, handling unknown inputs, and integrating external knowledge, leading to inaccurate and contextually inappropriate responses.
Innovation Solution
Enhance LLMs with tools that grant access to external knowledge sources, such as knowledge graphs and APIs, and deploy them as agents within AI systems to provide contextually relevant responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LLMs are enhanced with external knowledge sources and multiple processing tools, then response accuracy and contextual relevance are improved, but system complexity and computational resources increase
Solution Approach 1:
The system segments the knowledge processing task into distinct components: a retrieval component that searches external knowledge sources, a ranking component that scores retrieved passages, and an LLM component that generates responses. This segmentation allows each component to be optimized independently while managing overall system complexity.
Solution Approach 2:
The patent introduces an intermediary ranking system between the external knowledge sources and the LLM. This intermediary component retrieves and ranks relevant passages before presenting them to the LLM, thereby improving response accuracy without requiring the LLM to directly process all external knowledge, thus managing complexity.
2Measurement precision
If a large number of text passages are retrieved and processed, then response relevance is improved, but processing time and computational cost increase
Solution Approach 1:
The system retrieves a larger number of passages than strictly necessary (excessive action) and then ranks them to select only the most relevant subset for processing. This approach ensures that the LLM receives sufficient contextual information for accurate responses while avoiding the time cost of processing all retrieved passages equally.
Solution Approach 2:
The ranking component performs preliminary filtering and scoring of retrieved passages before they are presented to the LLM. This preliminary action identifies and prioritizes the most relevant passages, reducing the processing time required by the LLM while maintaining response relevance.
3Measurement precision
If embedding-based similarity comparison is used for all text passages, then retrieval accuracy is improved, but computational resources and processing time increase
Solution Approach 1:
The system segments the computational workload by using embedding-based comparison only for the retrieval phase to identify candidate passages, then applying a more efficient ranking mechanism for the subsequent filtering phase. This segmentation reduces overall computational resource usage while maintaining retrieval accuracy.
Solution Approach 2:
The system applies embedding-based comparison to retrieve a superset of potentially relevant passages, then uses a lighter computational ranking approach to filter down to the most relevant subset. This partial application of the computationally intensive embedding method reduces overall energy consumption while preserving retrieval accuracy for the final selected passages.
Data Source
AI summary
Techniques are disclosed herein for selecting document chunks that are most relevant to a query. The techniques include receiving a query and comparing a plurality of stored text passages to the query using a first similarity metric. Based on the comparison, a subset of the plurality of stored text passages that are most similar to the query are selected. A plurality of sentences from the subset of the plurality of stored text passages are identified. The identified sentences are ranked based on the query and a second similarity metric. A subset of the sentences are selected based on the ranking. The subset of the sentences or a derivative thereof are output in response to the query.