Abstractive Summarization Model Entity Coverage Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing document summarization techniques often produce unfaithful summaries by 'hallucinating' information not present in the original document, leading to misrepresentation of content.
Innovation Solution
Fine-tuning abstractive summarization models using a corpus of article-summary pairs with pseudo labels to improve faithfulness, where entity coverage precision metrics are used to generate control codes that condition the model to prioritize content from the source document.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Length of moving object
If abstractive summarization models are trained to generate compact summaries, then summary brevity is improved, but faithfulness deteriorates due to hallucination
Solution Approach 1:
The patent applies preliminary action by pre-training the summarization model on a large corpus of article-summary pairs before fine-tuning on domain-specific data. This pre-training stage establishes a foundation for accurate information extraction and representation, enabling the model to maintain faithfulness while generating compact summaries in downstream applications.
Solution Approach 2:
The patent implements feedback mechanisms through the use of pseudo labels generated from entity coverage precision metrics. These pseudo labels serve as feedback signals during fine-tuning, allowing the model to learn from its own performance and adjust its generation to improve faithfulness while maintaining summary compactness.
2Reliability
If models are fine-tuned to improve faithfulness using entity coverage precision metrics, then hallucination is reduced, but computational complexity increases
Solution Approach 1:
The patent uses disposable pseudo labels generated from entity coverage precision metrics as a cost-effective alternative to complex verification mechanisms. These pseudo labels provide sufficient feedback for fine-tuning without requiring computationally intensive validation processes, thus improving faithfulness while keeping training complexity manageable.
Solution Approach 2:
The patent changes parameters by transforming the training objective to incorporate entity coverage precision metrics as pseudo labels. This parameter change allows the model to learn faithfulness constraints through standard fine-tuning procedures rather than requiring complex custom training algorithms, reducing computational complexity while improving reliability.
Data Source
AI summary
Embodiments described herein provide document summarization systems and methods that utilize fine-tuning of pre-trained abstractive summarization models to produce summaries that more faithfully track the content of the documents. Such abstractive summarization models may be pre-trained using a corpus consisting of pairs of articles and associated summaries. For each article-summary pair, a pseudo label or control code is generated and represents a faithfulness of the summary with respect to the article. The pre-trained model is then fine-tuned based on the article-summary pairs and the corresponding control codes. The resulting fine-tuned models then provide improved faithfulness in document summarization tasks.