Hallucination Scoring via Prompt-Answer Embedding Similarity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The hallucination phenomenon in large language models (LLMs) generates probability-based texts that may be grammatically or semantically appropriate but not accurate, leading to ethical issues and the spread of misinformation.
Innovation Solution
A method and apparatus for determining a hallucination score in a language processing system by receiving a prompt and an answer, inserting a keyword, generating word sets from the prompt and answer, calculating embedding vectors, and determining a hallucination score based on the similarity between these vectors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If LLM generates probability-based text, then the text appears grammatically or semantically appropriate, but the text may not be accurate (hallucination)
Solution Approach 1:
The patent introduces an intermediary verification system that compares generated text against the original prompt using embedding vectors. This mediator detects hallucinations by measuring semantic similarity between prompt elements and generated content, allowing the system to maintain fluent generation while filtering out inaccurate information through the verification layer
Solution Approach 2:
The patent implements a feedback mechanism where the hallucination verification results are used to evaluate and potentially correct generated text. The system calculates hallucination scores based on embedding comparisons and provides feedback on text reliability, enabling continuous improvement of generation accuracy while preserving fluency
2Reliability
If training data and reinforcement learning are used to solve hallucination, then model reliability improves, but the solution is not fundamental and has limitations
Solution Approach 1:
The patent replaces the complex mechanical training process with a computational verification approach using embedding vectors. Instead of retraining the model through extensive data and reinforcement learning, the system uses semantic similarity calculations to detect and evaluate hallucinations, significantly reducing training complexity while maintaining reliability improvement
3Measurement precision
If embedding vectors are calculated to determine hallucination score, then text reliability can be assessed, but computational complexity increases
Solution Approach 1:
The patent extracts only the essential semantic information from the prompt and generated text into embedding vectors, focusing on key elements for comparison. By taking out only the necessary semantic features rather than processing entire text structures, the system achieves precise hallucination detection while reducing unnecessary computational complexity
Data Source
AI summary
Disclosed herein are a method and apparatus for determining a hallucination score of an artificial intelligence model in a language processing system. The method for calculating a hallucination score includes receiving a prompt and an answer, inserting a keyword into the answer, generating a first word set by using words present in the prompt, generating a second word set by using words present in the answer with the inserted keyword, generating embedding vectors of the first word set and the second word set, and calculating a hallucination score based on the embedding vectors.