Hybrid RAG Retrieval With Knowledge Graphs for Rich Document Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing retrieval-augmented generation (RAG) systems for large language models (LLMs) in video conferencing are limited by context window size, lack mechanisms for handling long-term dependencies, and fail to integrate structured knowledge effectively, leading to incomplete or inaccurate responses.
Innovation Solution
A hybrid RAG approach that combines probabilistic ranking, embedding-based retrieval, and structured data incorporation using a knowledge graph to enhance LLM responses, enabling long-term dependencies and improved semantic understanding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional RAG systems are used with LLMs, then the system can process queries using available context, but the context window size limits the amount of information that can be retrieved and processed
Solution Approach 1:
The patent segments the context window into multiple sessions, where each session has its own context window. The system divides retrieved documents and information across multiple sessions, allowing the LLM to process information in manageable chunks while maintaining access to broader information through session memory and knowledge graphs.
Solution Approach 2:
The patent adds temporal and structural dimensions to the retrieval system by implementing session-based contexts with knowledge graphs that capture entity relationships across sessions. This multi-dimensional approach allows the system to retrieve and organize information beyond the linear constraints of a single context window.
2Duration of action of stationary object
If traditional RAG systems are used, then the system can handle simple queries, but it lacks mechanisms for handling long-term dependencies across multiple sessions
Solution Approach 1:
The system performs preliminary actions by pre-processing retrieved documents into structured formats and pre-building knowledge graphs that capture entity relationships before queries are submitted. This preparation enables the LLM to efficiently access and utilize long-term dependencies across sessions without requiring extensive processing during query execution.
Solution Approach 2:
The patent introduces session memory and knowledge graphs as intermediary structures between the retrieval system and the LLM. These intermediaries store and organize information across sessions, maintaining contextual relationships and enabling the LLM to access long-term dependencies while improving response accuracy through structured information presentation.
3Adaptability or versatility
If traditional RAG systems are used, then the system can retrieve documents, but it fails to integrate structured knowledge effectively
Solution Approach 1:
The patent transforms retrieved unstructured documents into structured knowledge representations by changing the parameter format from raw text to organized entities, relationships, and attributes. This transformation enables the system to integrate structured knowledge effectively, allowing the LLM to query and utilize information with higher precision through defined schemas and relationship types.
4Quantity of substance
If the system retrieves more documents to improve completeness, then more information is available, but retrieval accuracy and relevance decrease
Solution Approach 1:
The system segments the retrieval process into multiple stages with different ranking strategies. Initial retrieval fetches a broader set of documents, then subsequent ranking and filtering stages segment and prioritize these documents based on relevance criteria, allowing the system to process large volumes while maintaining high accuracy through progressive refinement.
Solution Approach 2:
The patent introduces intermediate ranking and filtering mechanisms that act as mediators between bulk document retrieval and final selection. These intermediaries apply multiple ranking criteria and relevance filters to large document sets, enabling the system to retrieve comprehensive information while maintaining high accuracy through layered evaluation processes.
Data Source
AI summary
Techniques for implementing hybrid retrieval augmented generation for rich document queries using a large language model (“LLM”) are disclosed. In an example method, a computing system receives documents which are added to databases. The computing system receives a query, from which tokens are generated. The computing system determines a first ranking of the documents using a probabilistic ranking method and a second ranking of the documents in response to a vector database query using an embedded representation. The computing system determines a third ranking of the documents based on the first ranking and the second ranking. The computing system selects top documents from the third ranking. The computing system determines a knowledge graph based on the top documents. The computing system outputs the query, the top documents, and at least a portion of the knowledge graph to an LLM. The computing system receives and outputs a response.