Abstractive Summarization Model Using Question Answer Rewards
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural abstractive summarization models often fail to capture critical facts in source documents and generate inconsistent information, leading to low recall and precision.
Innovation Solution
A computer-implemented method that uses question and answer rewards to improve abstractive summarization by training a sequence-to-sequence model in a Reinforcement Learning setting, evaluating summaries using multiple automatic measures and human judgments, and updating the generation model based on calculated rewards.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If abstractive summarization models generate concise summaries, then productivity is improved, but manufacturing precision deteriorates due to factual inaccuracies and inconsistency
Solution Approach 1:
The patent implements a feedback mechanism where generated summaries are evaluated against the source document to verify factual accuracy. The model receives feedback on inconsistencies and is retrained iteratively to improve precision while maintaining productivity.
Solution Approach 2:
The patent performs preliminary verification by checking generated summaries against the source document before final output. This preliminary action catches factual errors early, ensuring precision is maintained without significantly impacting productivity.
2Adaptability or versatility
If abstractive summarization models paraphrase source text, then adaptability is improved, but reliability deteriorates due to loss of critical facts
Solution Approach 1:
The system continuously monitors paraphrased output for factual consistency with the source document. When inconsistencies are detected, the model receives corrective feedback and adjusts its paraphrasing strategy to maintain reliability while preserving adaptability.
Solution Approach 2:
The patent employs dynamic adjustment of paraphrasing intensity based on the importance of factual accuracy for each summary. Critical facts are preserved with higher fidelity while less critical elements allow more creative paraphrasing, optimizing both reliability and adaptability.
Data Source
AI summary
In an approach to improve abstract summarization with question and answer rewards embodiments generate, by a question and answer generator, questions and answers corresponding to a generated summary. Further, embodiments evaluate received answers for the generated questions by analyzing received answers associated with the generated summary against answers received for an original summary, and calculate a reward based on the similarity between answers associated with generated summary and the original summary. Additionally, embodiments update the generation model by applying the calculated reward to further train the summary generation model.