Image-Sequence Object Discovery With Noise-Filtered Pseudo-Labels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing unsupervised object discovery methods struggle with noise in pseudo-labels and limited localization of static objects due to reliance on motion information, leading to robustness issues and incomplete object detection.
Innovation Solution
Introduces automatic noise filtering of pseudo-labels using confidence scores and distillation learning to integrate static object supervision, enhancing the model's ability to distinguish between noise and objects, particularly static ones.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If unsupervised object discovery is performed using motion information from optical flow maps, then human annotation is eliminated and training data can be obtained automatically, but the generated pseudo-labels contain noise and are incomplete, leading to reduced detection accuracy
Solution Approach 1:
The patent introduces an intermediary verification step between automatic pseudo-label generation and model training. A segmentation model first generates candidate object masks from optical flow maps, then these candidates are verified against the actual images to filter out noise and false positives. This intermediary verification process maintains the automatic generation advantage while significantly improving the reliability of training data.
2Extent of automation
If motion information is used to generate pseudo-labels for object discovery, then training can be performed without human annotation, but static objects are missed and the pseudo-labels are incomplete
Solution Approach 1:
The patent segments the object detection task into two distinct parts: moving objects detected through optical flow-based pseudo-labels, and static objects detected through a separate mechanism. The system generates pseudo-labels for moving objects while simultaneously identifying static objects that are absent from motion-based labels, ensuring both types of objects are included in the training data without requiring human annotation for either category.
3Productivity
If all generated pseudo-labels are used for training, then the training process is simple and fast, but noise segments are treated as valid objects, reducing model robustness
Solution Approach 1:
The patent implements a feedback mechanism where the segmentation model's predictions are evaluated against ground truth images to compute accuracy metrics. Pseudo-labels that fail to meet quality thresholds or are identified as noise segments are filtered out before being used for training. This feedback loop ensures that only high-quality pseudo-labels are incorporated, maintaining model robustness while preserving reasonable training efficiency.
Data Source
Figure 1a~1b
Figure 2
Figure 3~4
AI summary
Method for training a model (MDO, MDOEL) for discovering objects in an input image sequence (SI), the model comprising: an encoder (ENC, ENCEL) an attention module (ATT, ATTEL) configured to transform the first feature vector into a plurality of feature vectors called slots (S1, ..SK-1), a decoder (DEC) learning the attention maps (W1, ..WK-1) being supervised (404) by a set of binary masks for discovering moving objects produced by an external source, called pseudo-labels (PL), the pseudo-labels (PL) being filtered (FIL) by means of the steps of: determining (401) an attention map of the foreground of the image (Wfg), calculating (402) a confidence score from the average of the values of the attention map (Wfg) of the foreground of the image at the positions of each moving object present in a pseudo-label, filtering (403) the moving objects of the pseudo-labels for which the confidence score is lower than a predefined threshold,.