Instance Segmentation Model Training with Pseudo-Label Validation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training machine-learning models for object detection using partially labeled datasets is challenging due to the potential stagnation and generalization issues when relying on noisy pseudo-labels, especially when combining Semi-Supervised Learning (SSL) with Learning Through Interaction (LTI).
Innovation Solution
A method combining SSL and LTI synergistically generates pseudo-sequences using partially labeled datasets, leveraging temporal and spatial augmentations to reduce the need for specialized interaction sequences, and employs dynamic filtering of pseudo-labels to improve model generalization and precision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If SSL is used to train with partially labeled datasets, then the amount of labeled data required is reduced, but the model may stagnate due to learning from noisy pseudo-labels
Solution Approach 1:
The patent introduces an intermediary mechanism (temporal consistency verification and cross-view validation) between the noisy pseudo-labels and the model learning process. This intermediary filters and validates pseudo-labels by checking temporal consistency across multiple frames and cross-view agreement, preventing direct learning from potentially erroneous labels while still utilizing the unlabeled data.
Solution Approach 2:
The patent implements feedback loops where model predictions are continuously validated against temporal consistency and cross-view agreement. The validation results feed back into the pseudo-label selection process, dynamically adjusting which pseudo-labels are trusted and used for training, thereby improving reliability while maintaining data efficiency.
2Adaptability or versatility
If LTI is employed to improve model generalization, then the model learns temporal perception, but specialized datasets with annotated temporal frames are required
Solution Approach 1:
The patent creates synthetic temporal sequences by copying and augmenting static labeled images to generate pseudo-video frames. Instead of requiring actual annotated temporal datasets, the system generates synthetic temporal variations through spatial augmentations and temporal transformations, applying LTI principles to these synthesized sequences.
Solution Approach 2:
The patent performs preliminary actions by pre-processing static images into augmented sequences before training. Spatial augmentations, temporal augmentations, and synthetic sequence generation are performed in advance, preparing the data structure needed for LTI without requiring actual temporal annotations during the data collection phase.
3Adaptability or versatility
If noisy pseudo-labels are reinforced across entire sequences, then SSL and LTI are combined, but the quality of labels deteriorates
Solution Approach 1:
The patent applies local quality by differentiating the treatment of different parts of the sequence and different pseudo-labels. Instead of uniformly reinforcing all labels across sequences, the system selectively validates and reinforces only those pseudo-labels that demonstrate temporal consistency and cross-view agreement, while discarding or down-weighting inconsistent labels.
Solution Approach 2:
The patent employs partial action by selectively applying pseudo-label reinforcement only to validated labels that pass temporal and cross-view consistency checks. Rather than reinforcing all pseudo-labels excessively, the system applies partial reinforcement to high-confidence predictions, maintaining label quality while still utilizing the benefits of SSL and LTI.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for training an instance segmentation model includes: providing unlabeled images and labeled images representing labeled objects; generating a first image by including one or more of the labeled objects into an unlabeled image, generating a second image by including one or more additional labeled objects into the first image and/or removing at least one of the one or more labeled objects from the first image, generating a third image by spatially augmenting the first image; training the model by: generating a first prediction, a second prediction, and a third prediction by inputting the first image, the second image, and the third image, respectively, into the model; determining an embedding loss of the first prediction and the second prediction, determining a regularization loss of the first prediction and the third prediction, wherein the first prediction represents pseudo-labels, and training the model using the embedding loss and the regularization loss.