Document Extraction Model Retraining With Error Label Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models for automated data extraction from documents suffer from errors that can severely impact downstream applications, necessitating a method to reduce these errors effectively.
Innovation Solution
An error detection and correction framework is implemented using a machine learning model that iteratively identifies and corrects errors through an active learning setup, involving an error checking controller, label controller, and training controller to refine the model's performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If machine learning models are used for automatic data extraction, then productivity is improved, but reliability deteriorates due to extraction errors
Solution Approach 1:
The patent implements a feedback mechanism where extracted data is automatically verified against ground truth labels, and extraction errors are fed back into the training process. The training controller retrain the machine learning model using labeled erroneous data, creating a continuous feedback loop that improves reliability while maintaining high productivity through automation.
Solution Approach 2:
The patent performs preliminary error detection and labeling before the final extraction is considered complete. The error checking controller identifies erroneous data and the label controller generates corrections in advance, allowing the model to learn from mistakes before they impact downstream applications, thus improving reliability without sacrificing extraction speed.
2Reliability
If iterative training with error correction is implemented, then reliability is improved, but device complexity increases
Solution Approach 1:
The patent segments the error correction framework into distinct modular components: an error checking controller for identifying errors, a label controller for generating corrections, and a training controller for model retraining. This segmentation reduces overall complexity by making each component independent and manageable, while collectively improving reliability through systematic error correction.
Solution Approach 2:
The framework implements self-service through automated error detection and labeling. The error checking controller automatically identifies extraction errors without human intervention, and the label controller generates corrections autonomously. This self-service capability improves reliability through continuous self-improving training while minimizing the operational complexity of managing the correction process.
3Productivity
If automated error checking and labeling is implemented, then productivity is improved, but measurement precision deteriorates in error detection
Solution Approach 1:
The patent uses feedback from automated error checking to improve measurement precision. The error checking controller continuously monitors extraction results and feeds back precise error type information to the label controller and training controller. This feedback loop enables the system to maintain high productivity through automation while improving the precision of error detection through iterative learning from actual extraction errors.
Solution Approach 2:
The patent changes parameters in the training process to improve measurement precision. The training controller adjusts model parameters and training hyperparameters based on the distribution and characteristics of detected error types. This allows the system to maintain high automated processing throughput while improving the precision of error type classification by adapting the measurement criteria to the specific characteristics of extraction errors.
Data Source
AI summary
A method including extracting, by a machine learning model executing using an electronic document, data to create extracted data. An error checking controller is executed on the extracted data to identify erroneous data within the extracted data. A label for the erroneous data is generated by a label controller executing on the erroneous data. The label identifies a type of error of the erroneous data and a correction to the type of error. The label is added to the erroneous data to generate labeled erroneous data. A training controller executes iterative steps to train the machine learning model using the electronic document, the labeled erroneous data, and a first instruction to generate new extracted data. The trained machine learning model is returned. The trained machine learning model has a reduced data extraction error rate relative to the machine learning model prior to executing the training controller.