Class-Agnostic Object Mask Generation From Bounding Boxes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional instance segmentation techniques rely on human-generated mask annotations, which can be misaligned, and are domain-specific or require semantic information, limiting their accuracy and class agnosticism.
Innovation Solution
A vision transformer-based auto-labeling framework generates class agnostic object masks using only bounding box annotations, expanding the cropped image region to include background pixels, and employing a two-phase training approach to improve mask quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If human generated annotations are used for instance segmentation, then pixel-level object masks can be obtained, but the annotations may be misaligned compared with object boundaries resulting in poor quality labeled segmentation masks
Solution Approach 1:
The system uses automated mask generation algorithms that process bounding box annotations independently to create segmentation masks without human intervention. The algorithm expands bounding boxes to include background pixels and uses neural network parameters to predict binary masks, allowing the system to self-generate annotations rather than relying on potentially misaligned human annotations.
Solution Approach 2:
The system creates a copy of the bounding box annotation and processes it through expansion and neural network prediction to generate a new binary mask. This copied and transformed annotation serves as the basis for training instance segmentation models, replacing the need for direct human-generated pixel-level masks.
2Productivity
If conventional automatic mask generation techniques are used, then human annotation effort is reduced, but the techniques are domain specific and operate on full images or rely on semantic information limiting class agnosticism
Solution Approach 1:
The mask generation algorithm is designed to be domain-independent and class-agnostic, working with any type of object without requiring semantic information. The same bounding box expansion and neural network prediction process applies universally across different domains and object classes, making the system highly adaptable and versatile.
Solution Approach 2:
The system extracts only the necessary bounding box information from annotations and processes it through expansion and neural network prediction, removing the need for full-image processing or semantic information. This extraction approach enables class-agnostic mask generation that works across different domains.
3Ease of manufacture
If bounding box based mask generation is used, then annotation process is simplified, but the technique relies on semantic information to generate masks and is therefore not class agnostic
Solution Approach 1:
The system replaces semantic information processing with a mechanical bounding box expansion process followed by neural network prediction. Instead of using semantic understanding to generate masks, the system mechanically expands bounding boxes to include background pixels and uses learned parameters to predict binary masks, achieving class independence.
Solution Approach 2:
The system changes the approach from semantic-based mask generation to parameter-based generation using bounding box expansion ratios and neural network parameters. By adjusting expansion parameters and training the neural network on bounding box data, the system achieves class-agnostic mask generation while maintaining annotation simplicity.
Data Source
AI summary
Class agnostic object mask generation uses a vision transformer-based auto-labeling framework requiring only images and object bounding boxes to generate object (segmentation) masks. The generated object masks, images, and object labels may then be used to train instance segmentation models or other neural networks to localize and segment objects with pixel-level accuracy.