LLM Output Hallucination Detection Using Semantic Entity Pairs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generative language models often produce factually inaccurate or misleading responses, known as hallucinations, which are unacceptable in domains like aviation where accuracy is critical, and current methods to combat hallucinations require unsustainable human intervention.
Innovation Solution
A computing system that uses entity extraction, semantic pairing, and semantic similarity models to compare entities from generative AI output with source text, identifying hallucinations by forming semantic pairs and comparing them against a domain-specific knowledge base.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If generative language models are used to generate text output, then productivity is improved, but reliability deteriorates due to hallucinations
Solution Approach 1:
The patent introduces an intermediary verification system that includes an entity extraction model, semantic pairing model, and semantic similarity model. These components act as mediators between the generative LLM and the final output, extracting entities from both the generated text and source text, forming semantic pairs, and comparing them to detect hallucinations before the text is finalized.
2Reliability
If human intervention is increased to verify accuracy, then reliability is improved, but productivity deteriorates due to time-consuming manual review
Solution Approach 1:
The system implements self-service verification where the AI model automatically performs entity extraction, semantic pairing, and hallucination detection without requiring human reviewers. The processing circuitry autonomously compares semantic pairs from generated text against source text entities, enabling the system to self-verify its output accuracy while maintaining high productivity.
3Reliability
If entity extraction and semantic comparison processes are added, then reliability is improved through hallucination detection, but device complexity increases
Solution Approach 1:
The verification system is segmented into distinct functional modules: an entity extraction model that identifies entities from text, a semantic pairing model that creates meaningful entity pairs, and a semantic similarity model that compares pairs for hallucination detection. This segmentation allows each component to specialize in a specific task, improving overall reliability while making the complex system more manageable and maintainable.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
A computing system for identifying hallucinations in generative artificial intelligence (AI) output includes processing circuitry configured to receive a text output generated by a generative large language model (LLM) in response to an input prompt including origin source text data, and using an entity extraction model, extract entities from the text output and from the origin source text data. The processing circuitry, using a semantic pairing model, forms first semantic pairs from the entities of the text output and second semantic pairs from the entities of the origin source text data, and using a semantic similarity model, semantically compares the first semantic pairs with the second semantic pairs. The processing circuitry, based on at least the comparison, classifies whether or not any of the first semantic pairs is a hallucination, and outputs an indication of the classification.