LiDAR Range Image Segmentation for Multi-View 3D Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional autonomous vehicle perception systems using Deep Neural Networks (DNNs) face challenges in accurately detecting and classifying objects, particularly pedestrians and bicycles, due to limitations in predicting 3D bounding boxes and object orientation from perspective views, and struggle with distinguishing features not visible from the analyzed view.
Innovation Solution
A multi-view perception system using DNNs that processes LiDAR data by sequentially analyzing perspective and top-down views, extracting classification data from LiDAR range images, and projecting it into a top-down view to regress object geometry and orientation, enabling accurate detection and classification of objects like motorcycles, bikes, and pedestrians with high recall rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If panoptic segmentation DNNs are used to perform class and instance segmentation in perspective view, then detection of objects with visible features is improved, but accuracy in predicting 3D bounding boxes and object orientation deteriorates
Solution Approach 1:
The patent transforms the problem from 2D perspective view segmentation to 3D point cloud processing. By converting perspective images into 3D point clouds and using multi-view geometry, the system recovers depth information and 3D object properties that are lost in traditional 2D panoptic segmentation, thereby improving 3D bounding box and orientation prediction accuracy.
Solution Approach 2:
The patent introduces an intermediary representation called 'depth map' or 'point cloud' that bridges the gap between 2D image segmentation and 3D object understanding. This intermediary contains both the segmentation information from the perspective view and the depth/orientation information needed for accurate 3D bounding box prediction.
2Measurement precision
If BEV object detection DNNs are used to detect objects in top-down view, then single class detection is improved, but accuracy in distinguishing pedestrians and bicycles from similar objects deteriorates
Solution Approach 1:
The patent merges the advantages of both perspective view segmentation and top-down view detection by combining their outputs. The system performs segmentation in perspective view to capture detailed object features, then transforms this information to top-down view for accurate localization and classification, thereby distinguishing pedestrians and bicycles from similar objects like poles or tree trunks.
Solution Approach 2:
The patent uses multi-view geometry to transform information between 2D perspective views and 3D/ top-down views. By leveraging the complementary information from different viewing angles, the system maintains distinguishing features while achieving accurate top-down localization.
3Measurement precision
If 3D convolutions are used over 3D voxelized volume to improve object detection accuracy, then detection precision is improved, but computational efficiency deteriorates
Solution Approach 1:
The patent extracts only the necessary 3D information from the point cloud data rather than processing the entire 3D voxelized volume. By selectively extracting relevant features and using efficient point cloud processing algorithms, the system achieves high detection accuracy without the computational burden of full 3D convolutions over large voxel volumes.
Solution Approach 2:
The patent changes the representation parameters from dense 3D voxels to sparse point cloud representations. This parameter change reduces the computational complexity while maintaining the ability to perform accurate 3D object detection and segmentation.
Data Source
AI summary
A deep neural network(s) (DNN) may be used to detect objects from sensor data of a three dimensional (3D) environment. For example, a multi-view perception DNN may include multiple constituent DNNs or stages chained together that sequentially process different views of the 3D environment. An example DNN may include a first stage that performs class segmentation in a first view (e.g., perspective view) and a second stage that performs class segmentation and/or regresses instance geometry in a second view (e.g., top-down). The DNN outputs may be processed to generate 2D and/or 3D bounding boxes and class labels for detected objects in the 3D environment. As such, the techniques described herein may be used to detect and classify animate objects and/or parts of an environment, and these detections and classifications may be provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.