Pluralistic Salient Object Detection With Learnable Mask Tokens
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing salient object detection (SOD) technologies struggle with ambiguity in defining multiple salient objects due to annotation inconsistencies in datasets like DUTS, leading to suboptimal performance in generating single segmentation masks.
Innovation Solution
A pluralistic object detector generates multiple segmentation masks for images with multiple salient objects, using a trained model that learns tokens to produce distinct masks for different object combinations, and a quality predictor assigns scores without ground truth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a single segmentation mask is generated for each image, then the system maintains simplicity and computational efficiency, but it fails to resolve ambiguity when multiple salient objects are present
Solution Approach 1:
The patent segments the salient object detection task by introducing multiple learnable tokens (e.g., five tokens) that each represent different object scenarios. The model generates multiple segmentation masks corresponding to different token selections, allowing it to handle images with varying numbers of salient objects. This segmentation of the detection space enables the system to resolve ambiguity by providing specialized masks for different object configurations.
2Measurement precision
If multiple segmentation masks are generated for each image, then ambiguity resolution improves, but computational resources and processing time increase
Solution Approach 1:
The patent implements a quality predictor module that scores each generated segmentation mask to determine its quality. Instead of processing all possible masks equally, the system selectively processes and outputs only the highest-quality masks. This partial action approach ensures high segmentation accuracy for relevant cases while avoiding unnecessary computation on low-quality masks, thus maintaining productivity.
3Adaptability or versatility
If annotation inconsistencies are accepted in the training dataset, then data diversity and real-world representation improve, but model performance deteriorates due to ambiguous supervision signals
Solution Approach 1:
The patent transforms the static single-mask annotation paradigm into a dynamic multi-mask system. During training, the model learns to generate multiple segmentation masks corresponding to different object scenarios represented by different tokens. This dynamic approach allows the model to adapt to annotation inconsistencies by selecting the appropriate mask based on the input image characteristics, thereby maintaining both real-world coverage and model reliability.
Data Source
AI summary
Example solutions for pluralistic salient object detection are disclosed. A received image shows multiple objects, such as a first object and a second object. A pluralistic object detector is trained to learn tokens. When provided with the image and the first token, it generates a first segmentation mask corresponding to the first object, but not the second object, and when provided with the image and the second token, it generates a second segmentation mask corresponding to at least the second object (and possibly also the first image). When the pluralistic object detector is trained on five tokens, up to five different segmentation masks, each corresponding to a different selection of up to five objects, may be generated. Additionally, a quality predictor is disclosed that assigns quality scores to each of the different segmentation masks, without requiring ground truth for the image.