Model Training via Augmented Sample Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The accuracy of trained models in OCR and other machine learning tasks is limited due to inconsistencies in processing results for different augmented samples of the same original sample, leading to low accuracy and generalization.
Innovation Solution
A method is introduced that involves determining augmented sample sets for original samples, establishing constraints based on the differences in outputs from models of varying complexities, and training the model using these constraints to improve accuracy and consistency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a model is trained using traditional methods without constraints on augmented samples, then the training process is simple and fast, but the accuracy and generalization of the model deteriorate due to inconsistencies in processing results for different augmented samples
Solution Approach 1:
The patent introduces feedback mechanisms through constraint conditions that compare outputs of the same model on different augmented samples, and compare outputs of different models on the same samples. These feedback loops enable the training process to identify and correct inconsistencies, thereby improving model accuracy while maintaining manageable training complexity through structured constraint application.
Solution Approach 2:
The patent uses constraint conditions as intermediary elements that mediate between the model outputs and the training process. These constraints act as intermediaries that translate the requirement for consistency into actionable training signals, enabling improved accuracy without directly complicating the core training algorithm.
2Measurement precision
If a model with high complexity is used to generate constraints for training a low complexity model, then the accuracy and generalization improve, but the computational resources and training time increase
Solution Approach 1:
The patent employs a teacher-student model framework where a high-complexity teacher model generates constraint conditions that are then used to train a low-complexity student model. This copying approach allows the student model to inherit the accuracy benefits of the teacher model without requiring the student to compute the same complex constraints, thereby reducing computational resources and training time while maintaining improved accuracy.
3Adaptability or versatility
If the model is trained to produce consistent outputs for different augmented samples, then the generalization ability improves, but the training process becomes more complex and time-consuming
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing constraint conditions before the actual training process. The teacher model pre-generates the constraints based on augmented samples, and these pre-computed constraints are then used during student model training. This preliminary computation of constraints reduces the time required during training, as the model only needs to satisfy pre-established constraints rather than compute consistency in real-time, thereby improving generalization without proportionally increasing training time.
Data Source
AI summary
There is provided a method and apparatus of training a model, a device, and a medium, which relate to artificial intelligence, and in particular to a deep learning and image processing technology. The method may include: determining a plurality of augmented sample sets associated with a plurality of original samples; determining a first constraint according to a first model based on the plurality of augmented sample sets; determining a second constraint according to the first model and a second model based on the plurality of augmented sample sets, wherein the second constraint is associated with a difference between outputs of the first model and the second model for one augmented sample, and the first model has a complexity lower than that of the second model; training the first model based on at least the first constraint and the second constraint, so as to obtain a trained first model.