LLM Query Repair via Knowledge Graph and Correction Database
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) often generate responses that appear correct but are factually incorrect, despite further training, which can lead to inaccurate results.
Innovation Solution
A system that creates a knowledge graph from collected queries, augments it with statistical analysis and relationship information, and converts natural language statements into database queries using a knowledge graph language (KGL) to generate human-intelligible entities and repair results output by LLMs, ensuring accuracy and usability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If LLMs are trained to process natural language statements and generate responses, then the model can provide sophisticated responses, but the model often provides results that have the appearance of correct results but are factually incorrect
Solution Approach 1:
The patent introduces a knowledge graph as an intermediary between the LLM and the final output. The knowledge graph serves as a mediator that stores verified factual information and relationships, allowing the LLM to retrieve accurate data rather than generating it from scratch, thus resolving the contradiction between natural language processing capability and factual accuracy
Solution Approach 2:
The system implements a feedback mechanism where the LLM generates initial responses, which are then verified against the knowledge graph. Any discrepancies are detected and corrected by retrieving accurate information from the knowledge graph, creating a feedback loop that ensures factual accuracy while maintaining natural language interaction capabilities
2Reliability
If further training is applied to eliminate errors, then some incorrect results may be eliminated, but incorrect results may still appear
Solution Approach 1:
Instead of continuously training the LLM to eliminate errors, the patent creates a copy of accurate knowledge in the form of a knowledge graph. This knowledge graph is constructed from verified data and relationships, providing a static reference that prevents hallucinations without requiring continuous model retraining, thus improving reliability without increasing training complexity
3Measurement precision
If a knowledge graph is created from collected queries and augmented with statistical analysis, then accurate database queries can be generated, but the system complexity increases
Solution Approach 1:
The system segments the complex task of generating accurate database queries into distinct components: (1) collecting queries, (2) creating a knowledge graph from queries, (3) augmenting with statistical analysis, and (4) generating final queries. This segmentation allows each component to be optimized independently, improving query accuracy while managing system complexity through modular architecture
Data Source
AI summary
Query language statements are generated from natural language statements using a knowledge graph representing one or more databases. An LLM may be used to generate database language statements from natural language statements. The database language statements may be modified based on the knowledge graph. The database language statements may be corrected using a correction database. The correction database may include entries including a natural language statement, an original database language statement, and one or more corrections. Entries may be corrected in response to human corrections of outputs of the LLM. Entries with common corrections may be consolidated.