Knowledge-Constrained Tree Search Decoding for LLM Hallucination
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) suffer from hallucination, generating misinformation that is syntactically and semantically plausible but factually incorrect, and existing methods to mitigate this issue, such as fine-tuning or guided decoding, face challenges like computational expense, catastrophic forgetting, and inability to identify necessary knowledge for generation.
Innovation Solution
The proposed solution is a computer-implemented model-agnostic natural language generation method that uses a heuristic tree search process, specifically Monte Carlo tree search, coupled with a knowledge classifier to detect and reduce hallucinations. This method reorders token importance based on knowledge groundedness and replaces hallucinated tokens with new ones, generating text with the least hallucination and highest knowledge groundedness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If fine-tuning the language model with knowledge is performed to reduce hallucination, then hallucination is reduced, but computational cost increases and catastrophic forgetting occurs
Solution Approach 1:
The patent introduces a reader language model as an intermediary component that processes knowledge and guides the generation process without requiring fine-tuning of the main LLM. This mediator model handles the knowledge-grounding task, allowing the primary model to remain unchanged and avoid computational fine-tuning costs while still reducing hallucination through the intermediary's guidance.
Solution Approach 2:
The system segments the hallucination mitigation function into a separate reader language model component rather than integrating it into the main LLM through fine-tuning. This segmentation allows the knowledge-processing function to be handled independently, reducing the computational burden on the main model and preventing catastrophic forgetting of its original capabilities.
2Reliability
If fine-tuning the language model with knowledge is performed to reduce hallucination, then hallucination is reduced, but generalizability across tasks deteriorates due to catastrophic forgetting
Solution Approach 1:
The reader language model serves as an intermediary that handles knowledge-specific adaptations without modifying the main LLM's weights. This allows the main model to retain its generalizability across different tasks while the intermediary model adapts to specific knowledge domains, eliminating the catastrophic forgetting problem.
Solution Approach 2:
By segmenting the knowledge-adaptation function into a separate reader model, the system preserves the main LLM's original capabilities and generalizability. The segmented architecture allows task-specific knowledge processing without interfering with the main model's broad competencies.
3Use of energy by moving object
If guided decoding methods are used to mitigate hallucination without fine-tuning, then computational cost is reduced, but the method cannot identify necessary knowledge for generation leading to persistent hallucination
Solution Approach 1:
The reader language model acts as a knowledge-identifying intermediary that analyzes the input and determines which knowledge elements are relevant for generation. This intermediary component enables the system to identify necessary knowledge without expensive fine-tuning, then guides the decoding process to use only that identified knowledge, thereby reducing hallucination while maintaining computational efficiency.
Data Source
AI summary
Provided herein is a novel method named Knowledge-Constrained Tree Search Decoding (KCTS) is provided, which is a first discriminator-guided constrained decoding approach to knowledge-grounded generation. The KCTS method also employs a Monte-Carlo Tree Search decoding (MCTS) instead of conventional LM decoding methods, which re-orders each token's importance based on the estimated future faithfulness starting from that token with multiple simulations. Coupled with a novel token-level hallucination detection method, RIPA (Reward Inflection Point Approximation), trained to predict the starting point of the hallucinating token, enabling effective adaptation of the knowledge classifier defined on the sequence level to the token level, this novel method is able to greatly mitigate hallucination problems of LLMs without the need of fine-tuning, thereby preventing the incurrence of high computational costs.