Pseudo-Label Filtering for Accurate Machine Learning Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional auto-labeling methods for generating training data in machine learning models often result in inaccurate initial labels due to small or low-quality manually labeled data, leading to incorrect patterns being learned, especially in specialized domains like industrial sites, with incorrect labels accumulating and being difficult to verify manually.
Innovation Solution
A method using a combination of Vision Language Model (VLM) and Vision Foundation Model (VFM) to generate pseudo labels, followed by filtering and verification processes to ensure high-quality training data, including image-text alignment scores, bounding box sizes, and IOU thresholds to refine pseudo labels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional auto-labeling method is used with small or low-quality manually labeled training data, then the initial model training can be completed, but incorrect labels accumulate and mislabel detection becomes difficult at large scale
Solution Approach 1:
The patent introduces a mislabel detection algorithm as an intermediary component that operates between the auto-labeling process and model training. This algorithm detects incorrect pseudo labels by analyzing patterns and inconsistencies in the generated labels, preventing error accumulation without requiring manual verification of every label at scale.
Solution Approach 2:
The system implements a feedback mechanism where mislabel detection results are fed back into the auto-labeling process. When incorrect labels are detected, the system adjusts its labeling strategy or re-processes affected images, creating a closed-loop system that continuously improves label quality while maintaining automated operation.
2Reliability
If manual labeling is performed to ensure high quality labels, then label accuracy is improved, but time and cost increase significantly
Solution Approach 1:
Instead of manually labeling all training images, the system applies manual labeling only to a small subset of images for initial model training. The auto-labeling algorithm then generates labels for the remaining images, with mislabel detection providing quality assurance. This partial manual action achieves high overall accuracy without the time cost of complete manual labeling.
Solution Approach 2:
The system creates pseudo labels by copying the labeling pattern from a small set of manually labeled examples. The trained model learns from the high-quality manual labels and reproduces this labeling style automatically across the entire dataset, maintaining consistency and accuracy at scale.
3Productivity
If general VLM or VFM is used for auto-labeling, then large-scale data processing is enabled, but domain-specific details and rare events are missed
Solution Approach 1:
The patent applies local quality by using a domain-specific mislabel detection algorithm tailored to industrial inspection contexts. This algorithm is specifically designed to recognize domain-specific details and rare events that general models miss, providing enhanced detection precision for critical features while maintaining automated large-scale processing.
Solution Approach 2:
The system adjusts the parameters and configuration of the mislabel detection algorithm based on domain-specific requirements. By changing detection thresholds, feature weights, and analysis parameters to match industrial inspection criteria, the system achieves both large-scale processing capability and domain-specific precision.
Data Source
AI summary
There is provided a method for generating training data for training a machine learning model by a training data generating device, comprising: (a) in response to acquiring an original image, performing sub-processes of: generating an image caption for the original image, extracting at least one noun phrase from the image caption, and performing an open vocabulary object detection on the original image, to thereby generate at least one first pseudo label and performing sub-processes of: extracting at least one proposal from the original image, generating at least one region description, and generating at least one second pseudo label; and (b) filtering the at least one first pseudo label and the at least one second pseudo label according to a preset filtering condition, to thereby generate at least one integrated pseudo label, and generating the training data by annotating the original image with the at least one integrated pseudo label.