Answer Span Correction Using Delimited Training Tokens
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional machine reading comprehension (MRC) systems often produce partially correct answers when presented with answerable questions, lacking the ability to accurately validate and correct answer spans.
Innovation Solution
A method involving a computing device that generates multiple answers, constructs a new training set with delimiting tokens, and trains a new natural language question and answer system using an augmented training set, incorporating an AI model like a neural network to correct answer spans.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional MRC systems extract answers from context, then they can provide answer validation, but they produce partially correct answers instead of fully accurate answers
Solution Approach 1:
The system implements a correction model that receives feedback from the reader model's predictions and iteratively improves answer span accuracy. The correction model is trained on examples of incorrect predictions and their corresponding correct answers, enabling it to learn from errors and provide corrective feedback to improve overall system accuracy.
Solution Approach 2:
A correction model is introduced as an intermediary component between the reader model and the final answer output. This correction model acts as a mediator that refines the reader's predictions by identifying and correcting errors in answer span extraction, thereby improving the reliability of the final answer without changing the core reading comprehension functionality.
2Ease of manufacture
If MRC systems use standard training sets, then training is straightforward, but the systems cannot detect or correct errors in predictions
Solution Approach 1:
The system performs preliminary action by pre-processing the training data to identify and label incorrect answer spans before training the correction model. This preparatory step involves generating synthetic error examples and annotating them with correct answers, enabling the correction model to learn from these pre-prepared error patterns without requiring complex real-time error analysis during training.
Solution Approach 2:
The training process is segmented into two distinct phases: first training the reader model on standard MRC tasks, then training the correction model separately on error-correction examples. This segmentation allows each model to be optimized for its specific function while maintaining training simplicity, as each training phase uses dedicated data and objectives rather than attempting to combine multiple complex functions in one training process.
3Productivity
If answer spans are extracted directly without correction, then processing is fast, but the answer quality does not match ground truth
Solution Approach 1:
The correction model applies partial action by selectively correcting only the answer span portions that are likely to be erroneous, rather than re-processing the entire answer generation pipeline. This approach corrects specific error patterns (such as incorrect start/end positions) while preserving the overall efficient extraction process, achieving improved accuracy without proportionally increasing processing time.
Data Source
AI summary
A method of using a computing device to improve an answer generated by a natural language question and answer system includes receiving, by a computing device, multiple questions in a natural language question and answer system. The computing device further generates multiple answers to the multiple questions. The computing device still further constructs a new training set with the generated multiple answers, where each answer is compared with a corresponding question of the multiple questions. The computing device additionally augments the new training set with one or more tokens delimiting a span of one or more of the generated multiple answers. The computing device further trains a new natural language question and answer system with the augmented new training set.