LLM Prompt Augmentation via Semantic Knowledge Graph Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models face accuracy issues when relevant training data is not publicly available, leading to incorrect responses or failure to provide answers, and they may not be updated frequently, resulting in an information gap between training and usage.
Innovation Solution
The technique involves identifying entities in the user prompt, searching a semantic framework for information about these entities, and adding this information to the prompt as verbalized triples, which is then submitted to the large language model, allowing it to generate more accurate responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If large language models rely on publicly available training data, then the model can process information efficiently, but the model fails to provide accurate answers when relevant data is not publicly available or is proprietary
Solution Approach 1:
The patent introduces a knowledge graph as an intermediary between the user's query and the large language model. The knowledge graph stores proprietary information from multiple sources and retrieves relevant facts to supplement the model's training data, enabling accurate responses to questions about proprietary information without requiring the model to directly access private databases.
Solution Approach 2:
The system performs preliminary retrieval of relevant information from the knowledge graph before generating the model's response. This preliminary action ensures that the model receives supplementary factual information about proprietary topics before processing the query, improving accuracy without requiring post-processing corrections.
2Reliability
If large language models are updated frequently to incorporate new information, then the model remains current with latest data, but the training time and computational resources required increase significantly
Solution Approach 1:
The patent segments the information system into two parts: a static large language model that processes language efficiently, and a dynamic knowledge graph that stores and updates factual information. This segmentation allows the knowledge graph to be updated frequently with new proprietary information without requiring retraining of the computationally intensive language model, thus maintaining currency while reducing computational resource consumption.
Solution Approach 2:
The knowledge graph acts as an intermediary that bridges the static language model and the dynamic proprietary information sources. It retrieves relevant facts on-demand to supplement the model's responses, enabling the system to access current information without frequently retraining the language model.
3Quantity of substance
If the model provides detailed information about entities, then the response is more informative, but the model may hallucinate or provide incorrect information when data is not available
Solution Approach 1:
The system uses feedback from the knowledge graph to verify and supplement the language model's responses. When the model generates information about an entity, the retrieved facts from the knowledge graph serve as feedback to correct potential hallucinations and ensure accuracy, while still providing detailed information when the knowledge graph contains relevant data.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present disclosure provides techniques and solutions for automatically and dynamically supplementing user prompts to large language models with information to be used by the large language model in formulating a response. In particular, entities are identified in the original prompt. A semantic framework is searched for information about such entities, and such information is added to the original user prompt to provide a modified user prompt. In a particular example, the identified entities comprise triples, and verbalized triples are added to provide the modified user prompt. The modified prompt may be hidden from the user, so that a response of the large language model appears to be in response to the original prompt.