Multi-View LiDAR Perception for Accurate 3D Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional autonomous vehicle perception systems using Deep Neural Networks (DNNs) face challenges in accurately detecting and classifying objects, particularly pedestrians and bicycles, due to limitations in predicting 3D bounding boxes and object orientation from perspective and top-down views, leading to inefficiencies and reduced accuracy.
Innovation Solution
A multi-view perception system using DNNs that processes LiDAR data by sequentially segmenting and regressing object geometry in multiple views, combining perspective and top-down views to enhance object detection and classification, particularly for vulnerable road users like pedestrians and bicycles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional panoptic segmentation DNNs are used for perspective view object detection, then detection speed is maintained, but accuracy in predicting 3D bounding boxes and object orientation deteriorates
Solution Approach 1:
The system segments the object detection task into two distinct stages: a first stage that performs class segmentation in perspective view, and a second stage that performs class segmentation and instance geometry regression in top-down view. This segmentation allows each stage to specialize in specific detection aspects, improving overall accuracy while managing computational complexity through staged processing.
Solution Approach 2:
The system transforms the detection problem from a single perspective view to multiple views by introducing a top-down view in the second stage. This dimensional change from 2D perspective to top-down projection enables accurate prediction of 3D bounding boxes and object orientation by leveraging geometric relationships across different viewing angles.
2Measurement precision
If conventional BEV object detection DNNs are used for top-down projection, then computational efficiency is maintained, but accuracy in detecting pedestrians and bicycles deteriorates
Solution Approach 1:
The system segments the detection task so that the computationally intensive top-down view processing is performed only in the second stage for objects already identified in the first stage. This segmentation avoids processing all objects through both stages, thereby maintaining computational efficiency while improving detection accuracy for challenging objects like pedestrians and bicycles.
Solution Approach 2:
The first stage performs preliminary class segmentation in perspective view to identify candidate objects before the second stage performs detailed instance geometry regression in top-down view. This preliminary action filters out non-relevant objects early, reducing computational load in the second stage while ensuring accurate detection of vulnerable road users.
3Measurement precision
If 3D convolutions are used over 3D voxelized volume to improve detection accuracy, then object classification accuracy improves, but computational cost increases substantially
Solution Approach 1:
The system segments the 3D processing into two views: perspective view for class segmentation and top-down view for instance geometry regression. This segmentation avoids the need for computationally expensive 3D convolutions over voxelized volumes by utilizing 2D projections that capture sufficient geometric information for accurate object classification and bounding box prediction.
Solution Approach 2:
The system uses 2D projections (perspective and top-down views) as copies of the 3D scene that contain sufficient information for detection tasks. These 2D representations serve as efficient alternatives to full 3D voxelized volumes, maintaining detection accuracy while dramatically reducing computational cost by avoiding 3D convolution operations.
Data Source
AI summary
A deep neural network(s) (DNN) may be used to detect objects from sensor data of a three dimensional (3D) environment. For example, a multi-view perception DNN may include multiple constituent DNNs or stages chained together that sequentially process different views of the 3D environment. An example DNN may include a first stage that performs class segmentation in a first view (e.g., perspective view) and a second stage that performs class segmentation and/or regresses instance geometry in a second view (e.g., top-down). The DNN outputs may be processed to generate 2D and/or 3D bounding boxes and class labels for detected objects in the 3D environment. As such, the techniques described herein may be used to detect and classify animate objects and/or parts of an environment, and these detections and classifications may be provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.