Class-Agnostic Sub-Object Segmentation With Pseudo-Labels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional segmentation systems are limited to identifying sub-objects that have been learned during training, failing to segment unknown sub-objects of known or unknown objects without excessive user input.
Innovation Solution
A class-agnostic neural network architecture, such as a modified Mask2Former model, is used to segment sub-objects by clustering features and generating pseudo labels, allowing for automatic segmentation of specific objects into sub-objects without relying on ground truth labels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional segmentation systems are used, then training data and labeled sub-objects are required, but the system fails to segment unknown sub-objects without excessive user input
Solution Approach 1:
The system performs self-supervised learning by automatically generating pseudo-labels from unlabeled data through clustering algorithms. The model clusters features of sub-objects in feature space to create its own training labels without requiring human annotation, enabling it to segment unknown sub-objects autonomously
Solution Approach 2:
The system pre-processes unlabeled data by extracting features and clustering them to generate pseudo-labels before actual segmentation occurs. This preliminary labeling action enables the model to train on diverse sub-objects including unknown ones, improving its ability to segment objects without extensive user input
2Measurement precision
If class-specific segmentation models are used, then training accuracy is maintained, but the system cannot segment sub-objects not learned during training
Solution Approach 1:
The system employs a universal segmentation model that can handle both known and unknown sub-objects through class-agnostic clustering. The model learns generalizable features and clustering patterns that apply across different object types, making it universally applicable to segmentation tasks without requiring retraining for each specific object class
Solution Approach 2:
The system changes the approach from class-specific training to feature-space clustering, transforming the segmentation problem into a parameter-free clustering task. By clustering based on feature similarity rather than predefined classes, the model adapts to any sub-object type without requiring class-specific training data
3Ease of operation
If manual sub-object selection is performed, then user control over segmentation is maintained, but user interaction time and computing resources are consumed
Solution Approach 1:
The system provides feedback by automatically generating segmentation results that can be reviewed and adjusted by users. The pseudo-labeled segmentation serves as an initial output that users can verify or modify, reducing the time needed compared to manual selection while maintaining user control over the final segmentation
Data Source
AI summary
A method includes receiving an object mask of an object in an image. The method further includes generating a mask of a sub-object in the image using a machine learning model configured to receive the mask of the object. A first branch of the machine learning model predicts whether a pixel of the image belongs to a sub-object.