Scene Object Segmentation Using Prototype Masks for Overlap Handling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing real-time mobile object instance segmentation methods, particularly those using cascade or two-stage networks, are computationally expensive and struggle with accurately segmenting overlapping objects, leading to poor boundary quality and high execution times, making them impractical for mobile devices.
Innovation Solution
A lightweight neural network architecture that generates segment masks by determining object positions and shapes using a pre-trained model, incorporating center and prototype maps, and coefficient arrays to segment overlapping objects efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If cascade or two-stage networks are used for real-time mobile object instance segmentation, then segmentation quality is improved, but computational cost and execution time increase significantly
Solution Approach 1:
The patent segments the segmentation task into two independent parallel streams: a lightweight stream for fast inference and a heavy stream for high-quality segmentation. This allows the system to achieve both speed and quality by processing different aspects of segmentation simultaneously rather than sequentially
Solution Approach 2:
The patent merges the outputs from the lightweight stream and heavy stream through a fusion module that combines segmentation masks and confidence scores. This integration allows the system to leverage both fast and accurate segmentation results to produce final high-quality segmentation masks
2Measurement precision
If computation heavy blocks are used to produce good-quality instance segmentation masks, then segmentation quality is improved, but practical implementation in real-time mobile solutions becomes impractical
Solution Approach 1:
The patent divides the computational workload into two separate processing streams with different complexity levels. The lightweight stream uses simplified blocks for fast processing while the heavy stream uses computation-intensive blocks for quality, allowing mobile devices to run the lightweight stream in real-time
Solution Approach 2:
The patent applies different processing qualities to different parts of the segmentation task. The lightweight stream provides sufficient quality for most regions while the heavy stream enhances quality for complex regions, allowing the system to optimize computational resources based on local needs
3Reliability
If two-stage networks crop and resize regions of interest, then object detection is improved, but boundary quality deteriorates and execution time increases
Solution Approach 1:
Instead of cropping and resizing regions then resizing back (two-stage approach), the patent inverts the approach by maintaining full-resolution feature maps throughout processing and only performing selective operations on specific regions, thereby preserving boundary quality while reducing computational overhead
4Quantity of substance
If existing methods are used for overlapping objects, then some objects are detected, but segmentation masks for all objects including overlapping ones are not produced accurately
Solution Approach 1:
The patent employs dynamic processing where the system adaptively adjusts processing based on detected object characteristics. For overlapping objects, the system dynamically routes to appropriate processing streams and uses confidence-based fusion to resolve ambiguities, improving both detection quantity and segmentation accuracy
Data Source
AI summary
A method for segmenting objects in a scene by an electronic device is provided. The method includes inputting at least one input frame of the scene into a pre-trained neural network model, the scene including a plurality of objects; determining a position and a shape of each object of the plurality of objects in the scene using the pre-trained neural network model; determining an array of coefficients for pixels associated with each object of the plurality of objects in the scene using the pre-trained neural network model; and generating a segment mask for each object of the plurality of objects based on the position, the shape, and the array of coefficients for each object of the plurality of objects in the scene.