Machine Reading Comprehension Training with Masked Questions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine reading comprehension models are prone to being misled by interference sentences due to learning shortcuts, such as relying on interrogative words without utilizing the full input question information, leading to incorrect answers, and conventional methods to enhance robustness are inefficient and introduce noise.
Innovation Solution
A method and apparatus for training a machine reading comprehension model by jointly performing general and adversarial tasks, where the input question is partially or fully masked to prevent shortcuts, calculating losses, and updating the model to minimize total loss, thereby improving robustness without generating adversarial samples.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If automatically generated adversarial samples are added to improve robustness, then the model's anti-interference ability is improved, but unnatural sentences introduce noise and negatively impact model performance
Solution Approach 1:
The patent converts the harmful effect of adversarial samples (which introduce noise when artificially generated) into a beneficial training mechanism by using mask questions. Instead of adding external adversarial articles that may be unnatural, the method masks parts of the original question to force the model to rely on article content rather than question shortcuts, thereby improving robustness without introducing unnatural noise
Solution Approach 2:
Instead of adding adversarial samples to the article side, the patent inverts the approach by applying masks to the question side. This inversion allows the same robustness-training goal to be achieved through a different mechanism that avoids the noise problem of artificial adversarial generation
2Reliability
If iterative adversarial sample generation is used to improve anti-interference ability, then model robustness is improved, but multiple iterations of training are required making the method inefficient
Solution Approach 1:
The patent applies preliminary action by pre-defining mask templates and applying them directly to training questions without requiring iterative adversarial sample generation. The mask questions are prepared in advance based on the original training data, eliminating the need for multiple iterative training rounds while still achieving robustness improvement
Solution Approach 2:
The patent extracts the essential robustness-training mechanism from the complex iterative adversarial process by isolating the key element: forcing the model to answer masked questions. This extraction simplifies the training process to a single-pass approach using mask questions, removing the inefficient iterative generation component
Data Source
AI summary
A method and an apparatus for training a machine reading comprehension model, and a non-transitory computer-readable recording medium are provided. A training process is repeatedly performed using a training sample set to obtain a machine reading comprehension model. The training process includes inputting a sample article and a sample question into the machine reading comprehension model, generating a first predicted answer, and calculating a first loss between the first predicted answer and a sample answer; replacing the sample question with a mask to obtain a mask question, inputting the sample article and the mask question into the machine reading comprehension model, generating a second predicted answer corresponding to the mask question, and calculating a second loss between the second predicted answer and the sample answer; and updating the machine reading comprehension model so as to minimize a total loss.