LLM Hallucination Reasoning via Embedding Perturbation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large-Language Models (LLMs) generate nonsensical or inaccurate outputs due to hallucinations, as they fabricate plausible text despite lacking sufficient knowledge, making it difficult to determine the type of generated text and trustworthiness.
Innovation Solution
A method involving perturbation-based zero-shot hallucination reasoning, which includes perturbing embedding vectors with noise, computing divergence and negative log-likelihood, and using validation datasets to classify LLMs based on divergence and log-likelihood thresholds, identifying semantically meaningful tokens, and determining the type of hallucination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If LLMs generate text using probabilistic token prediction, then text generation flexibility and creativity are improved, but hallucination risk increases due to sampling low-probability tokens
Solution Approach 1:
The system performs preliminary actions by computing original token probability distributions and determining token position probability distributions before the actual text generation evaluation. This allows the system to establish baseline expectations for token sequences before perturbation analysis, enabling subsequent detection of hallucinations through divergence comparison.
Solution Approach 2:
The system introduces perturbation as an intermediary mechanism to detect hallucinations. By adding noise to embedding vectors and computing the resulting perturbed probability distributions, the system creates a mediator process that reveals inconsistencies between original and perturbed generations, thereby identifying hallucinated content without directly modifying the core generation process.
2Productivity
If LLMs fabricate plausible text when incapable of answering, then text completion capability is improved, but accuracy and truthfulness deteriorate
Solution Approach 1:
The system implements feedback by evaluating divergence between original and perturbed token probability distributions. When divergence exceeds thresholds, the system provides feedback indicating potential hallucinations, allowing users to assess the reliability of generated text. This feedback mechanism enables continuous monitoring of fabrication vs. genuine knowledge without reducing text completion capability.
3Measurement precision
If perturbation is applied to embedding vectors by adding noise, then hallucination detection capability is improved, but computational complexity increases
Solution Approach 1:
The system applies partial action by focusing perturbation analysis on specific tokens identified as potentially hallucinated, rather than uniformly analyzing all tokens. The system computes perturbed probability distributions selectively based on divergence thresholds and semantic meaningfulness criteria, reducing overall computational complexity while maintaining detection capability for critical hallucinations.
Data Source
AI summary
A method may include: receiving a prompt and generated text from the LLM; computing an original token probability distribution for each token in the prompt and in the generated text; receiving a token position probability distribution for each token position in the generated text from the LLM; identifying keywords in the prompt; perturbing embedding vectors for the keywords used by the LLM by adding noise to the embedding vectors; computing a perturbed probability distribution for the perturbed embedding vectors by providing the perturbed embedding vectors as an input to a neural network used by the LLM, wherein the neural network returns a perturbed token probability distribution; evaluating a divergence between the original token probability distribution and the perturbed token probability distribution; identifying semantically meaningful tokens in the generated text; calculating a mean of divergences for the semantically meaningful tokens; and classifying the LLM based on the mean of divergences.