Semi-supervised ML Training via Label Guessing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional machine learning model training methods require a high ratio of labeled data to unlabeled data, limiting their performance and robustness, especially in handling input variability such as occlusions, blurriness, and rotations.
Innovation Solution
A semi-supervised learning system that generates 'guessed' model outputs for unlabeled training inputs through data augmentation and processing, allowing the model to be trained with a lower ratio of labeled to unlabeled data, and using a combination of labeled and unlabeled loss terms to adjust model parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional machine learning training methods are used with limited labeled data, then training cost is reduced, but model performance and accuracy deteriorate
Solution Approach 1:
The system enables unlabeled data to serve itself by automatically generating pseudo-labels through the model's own predictions. The model processes unlabeled inputs, generates predicted outputs as pseudo-labels, and uses these self-generated labels for training, eliminating the need for external manual labeling while improving accuracy with limited labeled data
Solution Approach 2:
The system introduces pseudo-labels as an intermediary between the model and unlabeled data. These pseudo-labels act as a bridge that allows the model to learn from unlabeled data by translating it into a format similar to labeled data, enabling the model to utilize both labeled and unlabeled data effectively
2Measurement precision
If more labeled data is used to improve model accuracy, then model performance improves, but training time and computational resources increase
Solution Approach 1:
The system creates copies of labeled data through data augmentation techniques, generating multiple transformed versions of the same labeled input (rotations, flips, crops). This increases the effective training data size without requiring additional manual labeling, improving accuracy while avoiding the time cost of collecting and labeling more data
Solution Approach 2:
The system performs preliminary processing of unlabeled data by generating pseudo-labels before the main training process. This preliminary action prepares the unlabeled data in advance, allowing it to be efficiently utilized during training without adding significant time overhead to the overall process
3Device complexity
If conventional training methods are used with high labeled-to-unlabeled data ratio, then model training is simpler, but robustness to input variability deteriorates
Solution Approach 1:
The system dynamically adjusts the training process by iteratively generating new pseudo-labels and retraining the model. This dynamic approach allows the model to adapt to various input variations and improve robustness over time, while the automated nature maintains reasonable process complexity
Solution Approach 2:
The system creates a universal training approach that handles both labeled and unlabeled data through the same pseudo-labeling mechanism. This multi-functional approach processes diverse data types and variations uniformly, improving robustness to input variability while maintaining consistent training complexity
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a machine learning model. One of the methods includes receiving an unlabeled batch; receiving a labeled batch; generating, from the unlabeled batch and the labeled batch, a processed unlabeled batch and a processed labeled batch, the generating comprising: for each unlabeled training input of the plurality of unlabeled training inputs: generating, from the unlabeled training input, a plurality of augmented unlabeled training inputs; processing each of the augmented unlabeled training inputs using the machine learning model to generate a respective model output for each augmented unlabeled training input; generating, from the model outputs for the augmented unlabeled training inputs, a guessed model output; and associating the guessed model output with each of the augmented unlabeled training inputs; and training the machine learning model on the processed labeled batch and the processed unlabeled batch.