AI Summary Hallucination Detection Using Semantic Matching

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Large language models (LLMs) generate hallucinations, i.e., factually incorrect or nonsensical responses, in user-generated content, posing a challenge in ensuring the accuracy and credibility of AI-generated summaries.

Innovation Solution

A method and system using semantic matching operations and chain of summarization verification (CSV) techniques to detect and mitigate hallucinations by comparing AI-generated content with historical human-generated content, employing cosine similarity and pre-trained roBERTa embeddings to identify and correct textual perturbations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If LLM is used to generate content, then productivity is improved, but reliability deteriorates due to hallucinations

Engineering Contradiction:

Improvecontent generation speedVSAvoidfactually correct

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent introduces an intermediary verification system that acts as a mediator between the LLM content generator and the final output. This system includes a hallucination detection module that compares generated content against ground truth data, and a mitigation module that corrects identified hallucinations. The intermediary layer maintains the high productivity of LLM generation while ensuring reliability through automated verification and correction processes.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Measurement precision

If semantic matching operation is performed, then measurement precision is improved, but device complexity increases

Engineering Contradiction:

Improvehallucination detection accuracyVSAvoidsystem structure

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent replaces complex manual verification processes with automated semantic matching operations using natural language processing algorithms. The system employs pre-trained language models and embedding techniques to automatically compare generated content with ground truth data, substituting what would otherwise require human reviewers with sophisticated but automated computational processes. This reduces operational complexity while maintaining high detection precision.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS20250356198A1Method and system for detection and mitigation of artificial intelligence hallucinations

Publication Date: 2025.11.20 JPMORGAN CHASE BANK NA

AI summary

Methods and systems for detecting and mitigating hallucinations in artificial intelligence (AI) summarizations of user-generated content are provided. The method includes: receiving an AI-generated content item; retrieving historical content items that have been generated by human beings; comparing the AI-generated content item with the historical content items in order to determine whether text string matches are present; when a determination is made that no match exists, performing a semantic matching operation to identify text strings included in the historical content items that are semantically similar to text strings in the AI-generated content item; and determining, based on the comparison and the semantic matching operation, whether the AI-generated content item is a hallucination. When a hallucination is detected, the hallucination may be mitigated by removing a textual perturbation and/or replacing the textual perturbation with text that accurately reflects the original content item.