Instance Segmentation Training With Synthetic Image Triples
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training machine-learning models for object detection using partially labeled datasets is challenging due to the potential stagnation and generalization issues when using Semi-Supervised Learning (SSL) and Learning Through Interaction (LTI), which reinforces noisy labels across sequences.
Innovation Solution
A computer-implemented method combining SSL and LTI synergistically generates pseudo-sequences using partially labeled datasets, employing image triples with spatial and temporal augmentations to reduce the need for specialized interaction sequences, and uses pseudo-labels and dynamic filtering to improve model generalization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If Semi-Supervised Learning (SSL) is used to train on partially labeled datasets, then the cost of labeling is reduced, but the model stagnates and fails to generalize due to noisy pseudo-labels
Solution Approach 1:
The patent introduces an intermediary filtering mechanism that mediates between the noisy pseudo-labels from SSL and the model training process. This filter selectively validates pseudo-labels based on confidence thresholds and consistency checks, allowing beneficial pseudo-labels to contribute to training while blocking noisy ones, thus resolving the contradiction between using unlabeled data and maintaining model reliability
Solution Approach 2:
The patent implements a feedback loop where model predictions are continuously evaluated against ground truth for labeled images, and this feedback is used to adjust the filtering criteria for pseudo-labels. The system dynamically adapts the confidence thresholds and filtering stringency based on model performance, enabling the model to generalize better while still benefiting from the reduced labeling cost
2Reliability
If Learning Through Interaction (LTI) is used to improve model generalization, then the model learns temporal changes better, but specialized datasets with annotated temporal sequences are required which increases complexity
Solution Approach 1:
The patent creates synthetic temporal sequences by copying and augmenting existing labeled images to generate pseudo-temporal sequences. Instead of requiring actual annotated video sequences, the system generates multiple augmented versions of static images with simulated temporal transformations, thereby achieving LTI benefits without the complexity of specialized temporal datasets
Solution Approach 2:
The patent performs preliminary data preparation by pre-processing existing labeled images into a format suitable for temporal learning before training begins. This includes pre-computing augmented versions and organizing them into sequence structures, which eliminates the need for complex temporal annotation during the actual training process and reduces dataset complexity
3Stability of the object's composition
If noisy pseudo-labels are reinforced across entire sequences, then training consistency is maintained, but the model learns incorrect patterns and fails to generalize
Solution Approach 1:
The patent applies partial filtering to pseudo-labels, using only the confident and consistent portion of pseudo-labels for training while discarding the noisy portion. Rather than using all pseudo-labels to maintain consistency, or filtering excessively to ensure quality, the system finds an optimal partial usage threshold that balances training consistency with generalization performance
Solution Approach 2:
The patent segments the pseudo-labels into different quality levels (high confidence, low confidence, inconsistent) and applies different training strategies to each segment. High-confidence pseudo-labels are used with full weight, low-confidence ones with reduced weight, and inconsistent ones are discarded, thereby maintaining training consistency from reliable sources while preventing noise propagation
Data Source
AI summary
A method for training an instance segmentation model. The method includes: providing unlabeled images and labeled images representing labeled objects; generating a first image by including one or more of the labeled objects into an unlabeled image, generating a second image by including one or more additional labeled objects into the first image and/or removing at least one of the one or more labeled objects from the first image, generating a third image by spatially augmenting the first image; training the model by: generating a first, second, and third prediction, by inputting the first image, the second image, and the third image, respectively, into the model; determining an embedding loss of the first prediction and the second prediction, determining a regularization loss of the first prediction and the third prediction, wherein the first prediction represents pseudo-labels, and training the model using the embedding loss and the regularization loss.