Classification Model Pretraining for Small-Sample Convergence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current classification technologies require large amounts of manually labeled data for training, leading to high costs and low efficiency, and struggle with poor model convergence when using small sample sizes.
Innovation Solution
A multi-stage progressive training method that includes self-pretraining with coarse data and fine-tuning with labeled data, utilizing a feature extraction module and classification module to adjust parameters, increasing training efficiency and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large amount of manually labeled data is used for training, then the model classification accuracy is improved, but the training cost and time consumption increase significantly
Solution Approach 1:
The patent applies preliminary action by performing self-pretraining on the classification model using a large amount of unlabeled original sample content before fine-tuning with a small amount of labeled data. This preliminary training phase prepares the model with general features and knowledge, so that subsequent fine-tuning with limited labeled data can achieve high classification accuracy much faster than training from scratch with the same labeled data.
Solution Approach 2:
The patent uses copying by leveraging the pre-trained model parameters and knowledge acquired from training on unlabeled data, then copying this knowledge to the fine-tuning phase. The pre-trained model serves as a starting point that can be adapted to specific classification tasks with minimal labeled examples, avoiding the need to relearn basic features from scratch.
2Measurement precision
If a large amount of manually labeled data is used for training, then the model classification accuracy is improved, but the training cost increases
Solution Approach 1:
The patent applies self-service by enabling the classification model to train itself on unlabeled original sample content through self-pretraining without requiring manual labeling. The model autonomously learns features and patterns from the unlabeled data, eliminating the need for expensive human annotators to label large portions of the training dataset.
Solution Approach 2:
The patent uses copying by transferring the knowledge and parameters learned during self-pretraining on unlabeled data to the fine-tuning phase. This allows the model to achieve high accuracy with minimal labeled data, significantly reducing the cost associated with data annotation while maintaining or improving classification performance.
3Productivity
If a small quantity of samples is used for training, then the training efficiency is improved, but the model convergence becomes difficult and classification accuracy deteriorates
Solution Approach 1:
The patent applies preliminary action by conducting self-pretraining on unlabeled data before fine-tuning with a small labeled dataset. This preliminary phase ensures the model has already learned robust features and is well-converged on general patterns, so that subsequent training with limited labeled data can quickly converge to task-specific accuracy without suffering from the poor convergence issues of small-sample training from scratch.
Data Source
AI summary
This application discloses a classification model training method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product. The method is performed by an electronic device and includes obtaining one original sample content, one labeled sample content, and a classification model; selecting at least one target sample content from the original sample content and constructing an expected content type corresponding to the target sample content; classifying the target sample content to obtain an actual content type of the target sample content; adjusting parameters of the classification model based on the actual content type; classifying the labeled sample content by using the adjusted classification model to obtain an actual content type of the labeled sample content; and updating a parameter of the adjusted classification model according to the actual content type for classifying a content to be processed to obtain a content type.