Fast Instance Segmentation via Depth Map Sorting and Jump Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current instance segmentation techniques, such as those based on neural networks like Mask-RCNN, are too computationally expensive and time-consuming, taking around two seconds per frame, which is not suitable for real-time applications like autonomous driving.
Innovation Solution
A fusioned object detection and depth mapping approach that defines a bounding box, sorts the depth map, detects low and high frequency jumps, and compares these to determine the largest or closest object, using digital signal processing with a computation complexity of O(3n), achieving fast instance segmentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If neural network-based instance segmentation (e.g., Mask-RCNN) is used, then segmentation accuracy is improved, but computation time increases to around two seconds per frame
Solution Approach 1:
The patent segments the depth map into multiple depth ranges (e.g., near, mid, far zones) and processes each segment separately. This division allows the system to focus computational resources on relevant regions, reducing overall processing time while maintaining accurate object boundary detection through targeted pixel-wise classification in each depth segment.
Solution Approach 2:
The patent extracts and utilizes depth information from depth maps as a separate feature channel, removing the reliance on complex neural network processing of RGB images alone. By extracting depth features and combining them with RGB data at earlier processing stages, the system achieves accurate segmentation with significantly reduced computation time.
2Measurement precision
If traditional instance segmentation methods are used, then segmentation accuracy is maintained, but productivity decreases due to slow processing speed
Solution Approach 1:
The patent performs preliminary depth map processing, sorting, and object candidate identification before the main segmentation process. By pre-processing the depth information to identify potential object regions and depth discontinuities, the system reduces the computational burden on subsequent segmentation steps, thereby increasing processing speed without sacrificing accuracy.
Solution Approach 2:
The patent introduces depth information as an additional dimension beyond traditional 2D RGB image processing. By incorporating the third dimension (depth) from depth maps, the system creates a more informative feature space that enables faster and more accurate segmentation, as depth provides direct geometric cues about object boundaries and spatial relationships.
3Productivity
If computational complexity is reduced for real-time processing, then processing speed increases, but segmentation accuracy may deteriorate
Solution Approach 1:
The patent introduces depth map information as an intermediary feature that bridges RGB image data and segmentation outcomes. This intermediate depth representation provides explicit geometric constraints that guide the segmentation process, enabling accurate object boundary detection with simpler, faster algorithms that don't require heavy neural network computation.
Solution Approach 2:
The patent changes the parameter space by incorporating depth values as an additional channel in the feature representation. This parameter expansion from 3-channel RGB to 4-channel (RGB-D) data provides richer information that simplifies the segmentation task, allowing for faster processing with maintained or improved accuracy through more discriminative feature inputs.
Data Source
AI summary
A method of instance segmentation, comprising, defining a bounding box, depth mapping the bounding box, sorting the depth map, estimating stability due to noise of the sorted depth map, detecting low frequency jumps of the sorted depth map based on the estimated stability, detecting high frequency jumps of the sorted depth map, comparing the detected low frequency jumps and the detected high frequency jumps, detecting at least one of a largest object and a closest object based on the comparison and pixel masking the at least one of the largest object and the closest object.