Neural Network Object Segmentation with Bounding Shape Guidance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing pipelines for object masking in machine learning models are computationally intensive and lack precision and recall, often incorrectly assigning masks to non-target objects.
Innovation Solution
The use of bounding shape information, such as boxes or polygons, is integrated into neural networks to guide and modify the mask determination process, allowing for improved precision and recall in object segmentation without requiring extensive training data on segmentation masks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional image processing pipelines are used for object masking, then segmentation can be performed, but computational resources (processor and memory) are significantly consumed
Solution Approach 1:
The system performs preliminary object detection to obtain bounding boxes before mask determination. This preliminary action provides spatial constraints that guide the subsequent mask generation process, reducing the computational search space and resources required for precise mask determination while improving efficiency
Solution Approach 2:
The system applies different processing strategies to different regions: using bounding box constraints for spatial localization and neural networks for detailed mask generation only within relevant regions. This local differentiation reduces overall computational resources while maintaining precision where needed
2Productivity
If conventional image processing pipelines are used for object masking, then segmentation can be performed, but precision and recall are insufficient with masks incorrectly assigned to non-target objects
Solution Approach 1:
The system uses bounding boxes to define local regions of interest, applying mask determination only within these constrained spatial boundaries. This local quality approach ensures masks are generated precisely where objects are located, improving both accuracy and efficiency by avoiding processing of irrelevant regions
Solution Approach 2:
The bounding box serves as an intermediary between object detection and mask determination. It provides spatial constraints that guide the neural network to generate masks only within the correct object boundaries, preventing incorrect mask assignments to non-target objects while maintaining processing speed
Data Source
AI summary
In various examples, systems and methods are disclosed relating to using neural networks for object detection or instance/semantic segmentation for, without limitation, autonomous or semi-autonomous systems and applications. In some implementations, one or more neural networks receive an image (or other sensor data representation) and a bounding shape corresponding to at least a portion of an object in the image. The bounding shape can include or be labeled with an identifier, class, and/or category of the object. The neural network can determine a mask for the object based at least on processing the image and the bounding shape. The mask can be used for various applications, such as annotating masks for vehicle or machine perception and navigation processes.