LLM Hallucination Verification Using Source Passage Citations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models often generate text with inaccuracies, known as 'hallucinations', which require manual review for accuracy, and there is a need for improved techniques for hallucination detection and remediation in text generation systems.
Innovation Solution
A system that generates novel text at the passage level, incorporating citations to unique identifiers, and fact-checks against source passages using large language models, machine learning methods, and rules-based entity extraction to verify accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If large language models generate text autonomously, then productivity is improved, but reliability deteriorates due to hallucinations
Solution Approach 1:
The patent introduces an intermediary verification system that acts as a mediator between the LLM generator and final output. This verification layer checks generated text against source documents without requiring manual review, thus maintaining productivity while improving reliability through automated fact-checking
Solution Approach 2:
The system implements feedback loops where generated text is continuously verified against source materials. The verification results feed back into the generation process, allowing the system to self-correct hallucinations and improve factual accuracy while maintaining autonomous operation
2Reliability
If manual review processes are implemented to verify accuracy, then reliability is improved, but productivity deteriorates due to increased workload
Solution Approach 1:
The verification system performs self-service by automatically checking generated text against source documents without requiring human intervention. This maintains high factual accuracy while preserving productivity, as the system handles its own verification independently
Solution Approach 2:
The patent replaces the mechanical manual review process with an automated computational verification system. This substitution maintains reliability through thorough checking while eliminating the productivity loss associated with human review time and workload
3Reliability
If comprehensive fact-checking is performed on all generated text, then reliability is improved, but device complexity increases
Solution Approach 1:
The verification system is segmented into modular components that handle different aspects of fact-checking independently. This segmentation maintains comprehensive verification capability while reducing overall system complexity through organized, manageable modules
Solution Approach 2:
The system performs verification selectively rather than uniformly on all text. By applying verification only where needed based on risk assessment or content type, the system maintains high reliability for critical outputs while reducing complexity through targeted rather than comprehensive verification
Data Source
AI summary
Enumerated source text passages may be determined based on one or more source text documents. The enumerated source text passages may include source text passage identifiers uniquely identifying the passages. A novel text passage including novel text portions may be determined based on a query and the enumerated source text passages. One or more of the novel text portions may be verified by a large language model to produce text verification information. A novel text generation message including novel text generated by the large language model may be determined based on the text verification information and sent to a client machine.