Machine Reading Comprehension Model Training via Iterative Data Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine Reading Comprehension (MRC) models face accuracy issues when trained on input-output pairs that are not understood, leading to confusion and decreased output accuracy, especially when the order of training pairs is not optimal.
Innovation Solution
The approach involves training an MRC model on a generic dataset, followed by iterative training on 'easy' domain-specific data, where 'easy' questions are those the model can answer correctly, and then gradually moving to 'difficult' questions, using an accuracy score threshold to determine which data to retrain on, ensuring the model understands all domain-specific data without becoming confused.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If an MRC model is trained on input-output pairs that it cannot understand, then the training data coverage is improved, but the model accuracy deteriorates
Solution Approach 1:
The patent applies preliminary action by pre-processing the training data to identify and remove input-output pairs that the model cannot understand. Before training, the system evaluates each training pair to determine if the model can comprehend it, and only includes comprehensible pairs in the training set. This prevents the model from learning from confusing or incomprehensible data, thereby maintaining high accuracy while still achieving comprehensive training data coverage.
2Productivity
If training is performed on all domain-specific data at once, then the training efficiency is improved, but the model becomes confused and accuracy deteriorates
Solution Approach 1:
The patent applies segmentation by dividing the domain-specific training data into multiple subsets based on the model's comprehension ability. The system segments data into different difficulty levels or comprehension categories, and trains the model iteratively on these segmented subsets rather than all data at once. This allows the model to progressively learn from easier to harder examples, maintaining accuracy while achieving efficient training through structured data organization.
3Ease of operation
If the order of input-output pairs is not optimized, then the training process is simpler, but the model accuracy deteriorates
Solution Approach 1:
The patent applies preliminary action by pre-sorting and optimizing the order of input-output pairs before training begins. The system evaluates the comprehensibility of each training pair and arranges them in an optimal sequence that progresses from most to least comprehensible for the model. This preliminary ordering ensures that the model encounters training data in a logical sequence that maximizes learning efficiency and accuracy, while the training process itself remains straightforward.
Data Source
AI summary
The present disclosure provides an approach for training a machine learning model by first training the model on a generic dataset and then iteratively training the model on “easy” domain specific training data before moving on to “difficult” domain specific training data. Inputs of a domain-specific dataset are run on the generically-trained model to determine which inputs generate an accuracy score above a threshold. The inputs with an accuracy score above a threshold are used to retrain the model, along with the corresponding outputs. The retraining continues until all domain specific dataset has been used to train the model, or until no remaining inputs of the domain specific dataset generate an accuracy score, when run on the model, that is above a threshold.