Sem2Ins Instance Segmentation via Semantic Label Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing instance segmentation methods are inefficient due to high complexity and latency, which is critical for real-time applications like autonomous driving, as they focus more on accuracy without considering inference latency.
Innovation Solution
A novel model, Sem2Ins, leverages conditional Generative Adversarial Networks (cGANs) with deep supervision and a weighted-fusion layer to generate instance boundaries from semantic segmentation, enabling real-time instance segmentation by bridging the gap between semantic and instance segmentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If detect-and-segment approaches (e.g., Mask R-CNN) are used for instance segmentation, then segmentation accuracy is improved, but inference latency increases and real-time processing capability deteriorates
Solution Approach 1:
The patent segments the instance segmentation task into two independent stages: first performing semantic segmentation to obtain per-pixel class labels, then generating instance boundaries from the semantic segmentation results. This divides the complex detect-and-segment process into simpler, faster sub-tasks that can be processed in real-time while maintaining accuracy.
Solution Approach 2:
The patent performs semantic segmentation as a preliminary action before instance boundary generation. By first obtaining the semantic label map that identifies object regions, the system prepares the groundwork for subsequent boundary detection, enabling faster processing compared to direct instance segmentation approaches.
2Measurement precision
If detect-and-segment approaches are used for instance segmentation, then segmentation accuracy is improved, but model complexity increases
Solution Approach 1:
The patent divides the complex instance segmentation model into two simpler components: a semantic segmentation network and a boundary generation network. Each component has reduced complexity compared to full instance segmentation models, while their combination achieves comparable or superior performance.
Solution Approach 2:
The patent introduces semantic segmentation results as an intermediary representation between the input image and final instance masks. This intermediate semantic label map simplifies the processing required for instance boundary generation, reducing overall model complexity.
Data Source
AI summary
The present disclosure relates to a system and a method for performing instance segmentation based on semantic segmentation that is capable of (1) processing HD images in real-time given semantic segmentation; 2) delivering comparable performance with Mask R-CNN in terms of accuracy when combined with a widely-used semantic segmentation method (such as DPC), while consistently outperforms a state-of-the-art real-time solution; (3) working flexibly with any semantic segmentation model for instance segmentation; (4) outperforming Mask R-CNN if the given semantic segmentation is sufficiently good; and (5) being easily extended to panoptic segmentation.