Dynamic Pseudo-Label Thresholding for Semi-Supervised Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional semi-supervised learning techniques face inefficiencies due to reliance on fixed thresholds for pseudo-label generation, leading to wasted unlabeled data, data imbalance, and sensitivity to hyperparameters, which prolong training time and affect model accuracy.
Innovation Solution
The method dynamically adjusts pseudo-label thresholds based on confidence scores from labeled and unlabeled data sets, allowing for category-specific adjustments and efficient use of unlabeled data to improve model recognition accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a fixed high threshold is used for pseudo-label generation, then the quality of pseudo-labels is improved, but a large amount of unlabeled data is wasted leading to longer training time
Solution Approach 1:
The patent applies dynamics by transitioning from a fixed threshold to a dynamic threshold that adapts during training. The threshold is initialized high to ensure quality pseudo-labels in early stages, then gradually lowered to utilize more unlabeled data as the model improves, resolving the contradiction between maintaining quality and utilizing sufficient data volume.
Solution Approach 2:
The patent changes the threshold parameter from a static value to a time-varying parameter that evolves during training. By adjusting the threshold based on training progress and model confidence, the system maintains high-quality pseudo-label generation while progressively incorporating more unlabeled data, thereby reducing training time without sacrificing quality.
2Ease of operation
If the same threshold is used for all categories, then the system is simple to operate, but data imbalance occurs for categories of different difficulty
Solution Approach 1:
The patent applies local quality by assigning different thresholds to different categories based on their difficulty levels. Easy categories receive higher thresholds for quality control, while difficult categories receive lower thresholds to ensure sufficient data availability, thereby resolving the data imbalance problem while maintaining manageable system complexity.
3Measurement precision
If model capacity is increased continuously to improve accuracy, then model accuracy is improved, but training data saturation occurs and further improvement becomes difficult
Solution Approach 1:
The patent applies dynamics by implementing a dynamic threshold adjustment mechanism that adapts to the model's learning progress. As the model capacity increases and approaches saturation, the threshold dynamically adjusts to optimize the utilization of unlabeled data, preventing further accuracy improvement bottlenecks and maintaining learning efficiency.
Data Source
AI summary
An optimizing method of semi-supervised learning and a computing apparatus are provided. In the method, a first predicted result of a labeled data set and a second predicted result of an unlabeled data set are respectively determined through a machine learning model. A pseudo-label threshold is determined according to a first confidence score of the first predicted result of a first sample of the labeled data set. The machine learning model is updated according to a compared result of the second predicted result of a second sample of the unlabeled data set and the pseudo-label threshold.