Neural Network Summarization Model Factuality Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing natural language processing summarization models often produce non-factual outputs, known as hallucinations, due to the lack of high-quality factual training datasets, which reduces the trustworthiness of their outputs.
Innovation Solution
A training framework that combines neural network-based language models with human feedback and annotated training samples, where summaries are reviewed and edited by humans to create a large, high-quality dataset of labeled factual and non-factual summaries, which are then used to fine-tune the models and train a factuality detector.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing summarization models are used, then natural language processing tasks can be performed, but the output factuality deteriorates due to hallucinations
Solution Approach 1:
The patent implements a feedback mechanism where human annotators review model-generated summaries and provide corrections. The corrected summaries are then fed back into the training dataset to retrain the model, creating a continuous improvement loop that progressively reduces hallucinations and improves output factuality
Solution Approach 2:
The patent performs preliminary factuality assessment by having human annotators review and correct summaries before they are added to the training dataset. This preliminary action ensures that only high-quality, factually accurate summaries are used for training, preventing the propagation of hallucinations
2Reliability
If human feedback is used to improve factuality, then output quality increases, but the complexity of the training process increases
Solution Approach 1:
The patent segments the training process into distinct modules: an initial summarization model, a factuality detector that identifies hallucinations, and a correction mechanism. This segmentation allows each component to be optimized independently while working together to improve overall factuality without overwhelming complexity
3Measurement precision
If more training data is collected through human annotation, then model performance improves, but the time and resources required increase
Solution Approach 1:
The patent implements a self-service approach where the system automatically generates initial summaries and identifies potential hallucinations using the trained factuality detector. This automation reduces the time and resources required for manual annotation while still collecting high-quality training data when human review is necessary
Data Source
AI summary
Embodiments described herein provide systems and methods for training neural network based language models using human feedback. An existing (or generated) summary of a document is provided, and that summary may be used to generate a number of other summaries. A human annotator may reject the summary if there is any factuality issue with the summary. Summaries which are agreed to have no factuality problems are used as baseline summaries. Small atomic edits are made to the baseline summaries (e.g., replacing a single word or phrase) to create a group of summaries. Human annotators label each of these summaries as factual or not. The annotated summaries are used to train a summarization model and/or a factual detector model.