Graph-Based Memory Extension for LLM Retrieval Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current Large Language Models (LLMs) face deficiencies in correctness, reliability, transparency, privacy, and flexibility, particularly when managing personal or proprietary data, and existing extensions fail to effectively utilize relational information due to reliance on similarity-based preselection of text chunks.
Innovation Solution
A computer-implemented method using a graph-based memory system that stores interconnected nodes and vectors, allowing for semantic proximity-based retrieval and activation of nodes to generate responses, incorporating relational information without the need for extensive retraining or fine-tuning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If similarity-based preselection of text chunks is used, then retrieval speed is improved, but retrieval accuracy deteriorates because complementary information is missed
Solution Approach 1:
The patent segments the retrieval process into two distinct phases: (1) a fast preselection phase using vector similarity to identify candidate text chunks, and (2) a refinement phase using graph-based semantic analysis to identify additional complementary information. This segmentation allows the system to maintain high retrieval speed while improving accuracy by addressing the limitations of similarity-based methods alone.
Solution Approach 2:
The patent introduces a graph-based memory structure as an intermediary between the vector embedding memory and the LLM. This graph structure stores semantic relationships and enables the system to retrieve not only semantically similar text chunks but also complementary information through graph traversal, thereby improving retrieval accuracy without sacrificing the initial fast preselection capability.
2Ease of operation
If LLMs use statistical likelihood completion, then natural language understanding is improved, but correctness and reliability deteriorate
Solution Approach 1:
The patent applies preliminary action by retrieving and preparing relevant context information from both vector embedding memory and graph-based memory before the LLM generates its response. This pre-retrieval of accurate, relevant information allows the LLM to operate within its strength (natural language understanding) while reducing reliance on statistical guessing, thereby improving correctness and reliability.
Solution Approach 2:
The system incorporates feedback mechanisms where the graph-based memory provides structured semantic relationships that guide and constrain the LLM's generation process. This feedback loop ensures that the LLM's statistical completion is grounded in retrieved factual information, improving the reliability and correctness of the final output.
3Adaptability or versatility
If conventional databases are combined with LLMs, then data management capability is improved, but system complexity increases
Solution Approach 1:
The patent implements a unified memory architecture where the graph-based memory structure serves multiple functions: it stores semantic relationships, enables complementary information retrieval, and provides structured context to the LLM. This multi-functional design improves data management capability while avoiding the need for entirely separate conventional database systems, thereby controlling system complexity.
4Adaptability or versatility
If proprietary memory is grown incrementally, then adaptability is improved, but retrieval quality may deteriorate without retraining
Solution Approach 1:
The patent implements a dynamic memory system where the graph-based memory structure can grow incrementally by adding new nodes and edges representing new knowledge. The graph traversal mechanisms adapt to this growing structure, maintaining retrieval quality by semantically navigating the expanded memory rather than relying on static pre-trained models. This allows the system to improve adaptability while maintaining retrieval quality through structural rather than statistical adaptation.
Data Source
AI summary
The disclosure concerns a method for generating a response to a query from a user or an agent using a vector embedding memory storing vectors and generated chunks of natural language text, and a graph-based memory storing information in the form of nodes interconnected by links. The method retrieves matching vectors from the vector embedding memory based on a semantic proximity of the stored vectors in the vector embedding memory with vectors generated based on the chunks of natural language text of the obtained query, determines and activates nodes stored in the graph-based memory based on a correspondence with the matching vectors and connected nodes based on links between the activated nodes, generates a response to the query based on chunks of text determined based on the semantic proximity, and based on determined chunks of text that correspond to the activated nodes as additional background information.