Single-Pass Panoptic Segmentation for Occluded Road Scenes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional autonomous vehicle perception systems struggle with heavy occlusions and complex scenes, often missing small and large objects, and have limited accuracy in predicting object classification, dimensions, and orientation due to the use of multiple DNNs and inflexible boundary regressors.
Innovation Solution
A deep neural network (DNN) performs panoptic segmentation with a single pass, using a common trunk and multiple heads to predict class confidence, instance regression, and instance clustering, improving scene understanding and accuracy by regressing instance location relative to corners rather than centroids.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple separate DNNs are used for detection and segmentation, then specific tasks can be performed with dedicated models, but the system becomes complex and slow
Solution Approach 1:
The patent combines multiple separate DNNs (instance segmentation DNN and boundary regressor DNN) into a single integrated DNN that performs both instance segmentation and boundary regression tasks simultaneously. This merging reduces system complexity and improves processing speed while maintaining detection accuracy through unified training and shared feature extraction
2Ease of manufacture
If conventional boundary regressors are used, then the system is simpler to implement, but they are inflexible and struggle with complex scenes
Solution Approach 1:
The patent implements a dynamic boundary regressor within the DNN that adapts to complex scenes by learning from training data. The boundary regressor is not static but is trained to handle various scene complexities including occlusions and irregular shapes, making the system flexible while maintaining implementation feasibility through integrated training
3Manufacturing precision
If individual parts are trained separately, then each component can be optimized independently, but the overall scene understanding is limited
Solution Approach 1:
The patent merges the training processes of instance segmentation and boundary regression into a unified training framework. The single DNN is trained jointly on both tasks, allowing the model to learn global scene characteristics and contextual relationships that improve overall scene understanding while still optimizing specific components through the shared loss function and feature extraction layers
Data Source
AI summary
A deep neural network(s) (DNN) may be used to perform panoptic segmentation by performing pixel-level class and instance segmentation of a scene using a single pass of the DNN. Generally, one or more images and/or other sensor data may be stitched together, stacked, and/or combined, and fed into a DNN that includes a common trunk and several heads that predict different outputs. The DNN may include a class confidence head that predicts a confidence map representing pixels that belong to particular classes, an instance regression head that predicts object instance data for detected objects, an instance clustering head that predicts a confidence map of pixels that belong to particular instances, and/or a depth head that predicts range values. These outputs may be decoded to identify bounding shapes, class labels, instance labels, and/or range values for detected objects, and used to enable safe path planning and control of an autonomous vehicle.