LLM Response Graph Scoring for Hallucination and Confidence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large Language Models (LLMs) exhibit a tendency to hallucinate and lack confidence in their responses, leading to inconsistent outputs and unreliable predictions.
Innovation Solution
A method and system for computing variability and confidence scores by constructing graphs based on paraphrase questions, documents, and responses using hardware processors, determining edge sets, and calculating scores based on cosine similarity and graph clustering to evaluate LLM performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If LLMs generate responses using probabilistic sampling, then response diversity and creativity are improved, but response consistency and reliability deteriorate
Solution Approach 1:
The patent implements a self-consistency mechanism where the LLM generates multiple responses to the same query and evaluates their consistency. The system uses a scoring function that compares semantic similarity between responses, providing feedback to identify and select the most reliable answer. This feedback loop enables the system to maintain reliability while preserving the creative benefits of probabilistic sampling.
Solution Approach 2:
The patent generates multiple responses (excessive action) beyond the single response traditionally provided by LLMs. By generating N responses where N>1, the system creates a pool of candidate answers that can be evaluated for consistency. This partial evaluation approach allows selecting the most reliable response without requiring all generated responses to be perfectly consistent.
2Measurement precision
If LLMs increase model size and complexity, then response quality and capabilities are improved, but computational cost and inference time deteriorate
Solution Approach 1:
The patent segments the response generation process into distinct phases: (1) generating multiple candidate responses, (2) evaluating consistency among responses, (3) selecting the most reliable answer. This segmentation allows the system to use a smaller, faster model for the consistency evaluation phase rather than requiring a large model for every operation, thereby reducing overall inference time while maintaining response quality.
Solution Approach 2:
The patent introduces a consistency evaluation mechanism as an intermediary between the LLM and the final output. This intermediary layer uses semantic similarity computation to assess response reliability without requiring the main LLM to be oversized. The intermediary enables efficient filtering of low-quality responses while preserving the capabilities of the core model.
3Adaptability or versatility
If LLMs use temperature sampling for diverse responses, then response variability is improved, but hallucination frequency deteriorates
Solution Approach 1:
The system uses self-consistency as a feedback mechanism to detect and filter hallucinations. When the LLM generates multiple responses with temperature sampling, the consistency evaluation identifies responses that diverge significantly from the consensus, which are likely hallucinations. This feedback allows the system to maintain high response variability while automatically filtering out unreliable generated content.
Data Source
AI summary
The rapid proliferation of Large Language Models (LLMs) across diverse organizations, domains, and modalities has revolutionized natural language processing applications. Despite their widespread adoption, a critical challenge persists: the inherent tendency of LLMs to hallucinate, exhibit substantial variability in responses, and often lack confidence in their predictions. Embodiments of the present disclosure provide system and method address the challenges associated with LLMs by identifying and selecting models for which various graphs such as query graph, response graph, and document graph are generated given one or more input queries and one or more documents. Various sets of edges are determined for computing variability score. Further, graph clustering is performed on response graph to compute a confidence score. The present disclosure enhances the reliability of LLM outputs, providing users with more consistent and trustworthy results across various applications.