LLM Answer Verification Through Source-Based Confidence Scoring

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Large Language Models (LLMs) often generate low-quality, misleading, or inaccurate outputs, including hallucinations, which can lead to misinformation and expose confidential information, due to their association of words with incorrect concepts during training.

Innovation Solution

A computing system is configured to determine confidence scores for LLM outputs based on the sources used, supplementing and annotating the answers with quality indicators, and implementing safeguards to ensure answers meet a quality threshold.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If LLMs generate text from massive datasets, then they can provide diverse and comprehensive answers, but they may learn incorrect associations leading to hallucinations and misinformation

Engineering Contradiction:

Improveanswer diversityVSAvoidanswer accuracy

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The patent introduces an intermediary verification system that acts as a mediator between the LLM's generated text and the final output. This system cross-checks the generated text against the original source documents to verify factual accuracy, thereby maintaining answer diversity while filtering out hallucinations and misinformation through an intermediate validation layer

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent implements a feedback mechanism where the generated text is continuously evaluated against ground truth source materials. The verification system provides feedback by identifying discrepancies between generated content and source information, allowing the system to correct inaccuracies and improve reliability while preserving the generative capability

Inventive Principle:

Principle #23Feedback

2Reliability

If the system evaluates and annotates all LLM outputs with quality scores, then answer reliability improves, but processing time and system complexity increase

Engineering Contradiction:

Improveanswer quality assuranceVSAvoidsystem complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent applies partial verification by focusing evaluation efforts on critical factual claims within the generated text rather than uniformly analyzing every word. The system identifies key assertions that require verification against source documents, performing targeted checks that ensure reliability while avoiding the excessive complexity of complete sentence-by-sentence analysis

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The patent dynamically adjusts verification parameters such as the depth of source checking and the stringency of quality thresholds based on the context and type of query. This allows the system to maintain high reliability for critical information while reducing processing complexity for less sensitive content, adapting the verification intensity to match the required quality level

Inventive Principle:

Principle #35Parameter changes

3Object-affected harmful factors

If the system withholds low-quality answers, then misinformation is prevented, but user productivity decreases due to fewer available answers

Engineering Contradiction:

Improvemisinformation preventionVSAvoiduser response rate

Core Design Contradiction:

Object-affected harmful factorsVSProductivity

Solution Approach 1:

The patent converts the potential harm of withheld answers into a benefit by using the verification process to identify and highlight high-quality, well-sourced information. Answers that pass verification are prominently displayed with quality indicators, turning the filtering mechanism into a value-added feature that helps users quickly identify reliable information among multiple responses

Inventive Principle:

Principle #22Blessing in disguise (Convert harm into benefit)

Solution Approach 2:

The patent segments the answer delivery process by separating verified high-quality answers from unverified or low-quality ones. The system presents verified answers with clear indicators of their reliability, allowing users to access confirmed information efficiently while still providing access to alternative answers for comprehensive research, thus maintaining productivity through structured information presentation

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS20250252270A1Generation of source-based confidence score for LLM output

Publication Date: 2025.08.07 WELLS FARGO BANK NA

AI summary

A computing system may be configured for generating a source-based confidence score in association with output from a Large Language Model (LLM). The computing system may obtain computer-generated text output from the LLM as an answer to an inquiry submitted by a computing device. The computing system may determine a confidence score in association with the answer to the inquiry based on an evaluation of one or more sources used by the LLM to generate the answer and determine whether the confidence score associated with the answer satisfies a quality threshold. Based on the confidence score associated with the answer satisfying the quality threshold, the computing system may generate an annotated answer including the answer and an indication of quality based on the evaluation of the one or more sources used by the LLM to generate the answer. The annotated answer may be output in response to the inquiry.