LLM Answer Grounding With Hallucination Detection and Regeneration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large Language Models (LLMs) used in domain-specific conversation systems suffer from the hallucination problem, generating untruthful information that can lead to reliability issues and potential harm, especially in applications requiring high accuracy, such as vehicle repairment operations and cooking assistants.
Innovation Solution
A method involving sentence similarity estimation and keyword matching is employed to detect and handle hallucinated content by comparing the generated answer with context information, and if detected, either removing or regenerating the answer using better context information, thereby improving reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If LLM generates answer based on training knowledge, then answer naturalness and intelligence improve, but hallucination risk increases
Solution Approach 1:
The patent introduces an information retrieval system as an intermediary between the user question and the LLM. The retrieval system fetches ground-truth context information from external sources, which then serves as the basis for LLM answer generation. This intermediary mechanism ensures that the LLM grounds its natural language generation in factual information, reducing hallucinations while maintaining answer quality.
Solution Approach 2:
The patent performs information retrieval before LLM answer generation. By pre-fetching relevant context information and verifying its availability, the system prepares the ground truth data in advance. This preliminary action ensures that when the LLM generates answers, it has access to verified information, thereby preventing hallucinations while preserving natural language capabilities.
2Reliability
If information retrieval is added to LLM-based QA system, then answer reliability improves, but hallucination still occurs occasionally
Solution Approach 1:
The patent implements a feedback mechanism where the system continuously monitors whether retrieved context information adequately answers the user question. If the retrieved information is insufficient or leads to hallucinations, the system feedbacks this information back to the retrieval module, triggering re-retrieval with adjusted parameters or alternative sources. This closed-loop feedback ensures that hallucinations are detected and corrected, maintaining high reliability.
3Adaptability or versatility
If LLM is trained on enormous training data, then NLP capabilities improve, but hallucination problem persists
Solution Approach 1:
The patent segments the knowledge source into two distinct components: (1) the LLM's internal training knowledge for general NLP capabilities, and (2) external retrieval-based ground truth information for domain-specific accuracy. By separating these knowledge sources and using retrieval for factual grounding, the system maintains the LLM's versatile NLP skills while eliminating hallucinations through external verification.
Data Source
AI summary
Systems and methods are described herein for detecting and reducing hallucinations and improving the reliability of an LLM-based QA system. Particularly, the disclosure provides a set of hallucination detection and handling approaches that address different types of hallucinations of the QA system. The hallucination detection approaches described herein include comparing a natural language answer with context information by way of sentence similarity estimation and keyword matching. The hallucination handling approaches described herein include removing hallucinated sentences from the natural language answer or regenerating the natural language answer using better context information, depending on a level of hallucination detected in the natural language answer. A hybrid framework is also provided that systematically combines the hallucination detection approaches and hallucination handling approaches into one system to achieve an optimal hallucination-reduction performance for the QA system.