Document–Abstract Consistency Detection With Adversarial Samples

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing abstract generation models face a trade-off between conciseness and factual accuracy, leading to inconsistencies between the abstract and the source document, necessitating improved methods for consistency detection.

Innovation Solution

A consistency detection model is trained using adversarial data augmentation to generate adversarial samples, applying interference information selectively to text elements in the abstract, ensuring the model can accurately identify inconsistencies within a predetermined threshold.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If the abstract is made more concise, then the productivity and efficiency of information delivery are improved, but the reliability and factual accuracy deteriorate due to higher probability of factual errors

Engineering Contradiction:

Improveinformation delivery efficiencyVSAvoidfactual accuracy

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent introduces an intermediary consistency detection model that acts as a mediator between the abstract generation process and the final output. This model independently verifies whether the generated abstract maintains factual consistency with the source document, allowing the system to produce concise abstracts while catching factual errors through the intermediary verification layer

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent implements a feedback mechanism where the consistency detection model evaluates generated abstracts and provides feedback signals. When factual inconsistencies are detected, the system can adjust the generation process or flag the abstract for review, creating a closed-loop system that improves reliability without sacrificing conciseness

Inventive Principle:

Principle #23Feedback

2Device complexity

If traditional consistency detection methods are used, then the device complexity is kept low, but the measurement precision and detection accuracy of factual errors are insufficient

Engineering Contradiction:

Improvedetection system complexityVSAvoidconsistency detection accuracy

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The patent replaces traditional mechanical or rule-based consistency detection methods with a neural network-based consistency detection model. This substitution enables the system to achieve higher measurement precision in detecting factual errors by learning complex patterns and relationships between documents and abstracts, moving beyond simple keyword matching or template-based approaches

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS12455913B2Method, device, and medium for consistency detection of a document and an abstract

Publication Date: 2025.10.28 BEIJING YOUZHUJU NETWORK TECH CO LTD

AI summary

A method of consistency detection of a document and an abstract is proposed. The method includes determining a first sample and first annotation information, the first annotation information indicating that a first abstract and a first document included in the first sample are inconsistent, at least one of a plurality of text elements in the first abstract being labeled as inconsistent with the first document; generating a first adversarial sample by applying interference information to the first sample, the interference information being applied to the first sample and other text elements in the first abstract except for the at least one text element; and training, at least based on the first sample, the first adversarial sample, and the first annotation information, a consistency detection model according to a training objective. The obtained trained model can better detect and track a part of the abstract that are inconsistent with the document.