Top-Down LiDAR Detection with Multi-View 3D Box Regression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional autonomous vehicle perception systems using Deep Neural Networks (DNNs) face challenges in accurately detecting and classifying objects, particularly pedestrians and bicycles, due to limitations in predicting 3D bounding boxes and object orientation from perspective and top-down views, leading to inefficiencies and reduced accuracy.
Innovation Solution
A multi-view perception system using DNNs that processes LiDAR data by sequentially segmenting and regressing object geometry in multiple views, including perspective and top-down views, to enhance object detection and classification, particularly for vulnerable road users like pedestrians and bicycles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional panoptic segmentation DNNs are used for perspective view object detection, then detection capability for certain classes like pedestrians and bicyclists is achieved, but accuracy in predicting 3D bounding boxes and object orientation is limited
Solution Approach 1:
The patent transforms the problem from 2D perspective view segmentation to 3D bounding box prediction by introducing a third dimension. The system predicts 3D bounding boxes (x, y, z coordinates, dimensions, and orientation) from 2D segmented instances, effectively adding spatial depth information to compensate for the limitations of perspective view analysis.
2Adaptability or versatility
If conventional BEV object detection DNNs are used for top-down projection detection, then multi-class detection capability is achieved, but accuracy in detecting pedestrians and bicycles is limited due to similar top-down appearances
Solution Approach 1:
The patent applies segmentation by first performing panoptic segmentation in perspective view to separate different object instances and classes, then using this segmented information to inform the BEV detection process. This preliminary segmentation helps distinguish pedestrians and bicycles from similar objects like poles and tree trunks before the top-down view analysis.
Solution Approach 2:
The system performs perspective view segmentation as a preliminary step before BEV detection. By pre-identifying and segmenting object instances in the perspective view with predicted 3D bounding boxes, the system prepares enhanced input information that improves the accuracy of subsequent BEV detection for challenging objects like pedestrians and bicycles.
3Measurement precision
If 3D convolutions are used to process LiDAR point clouds for BEV detection, then detection accuracy may be improved, but computational expense and processing of empty voxel space increase substantially
Solution Approach 1:
The patent extracts only the necessary information from the LiDAR point cloud by performing perspective view segmentation first, identifying object instances and their 3D bounding boxes. This extracted information is then projected to BEV, avoiding the need to process the entire 3D voxel space with computationally expensive 3D convolutions.
Solution Approach 2:
By performing perspective view segmentation and 3D bounding box prediction as a preliminary action, the system pre-processes the LiDAR data to identify relevant object regions. This preliminary processing reduces the amount of data that needs to be processed in the BEV detection stage, thereby reducing computational complexity while maintaining detection accuracy.
Data Source
AI summary
A deep neural network(s) (DNN) may be used to detect objects from sensor data of a three dimensional (3D) environment. For example, a multi-view perception DNN may include multiple constituent DNNs or stages chained together that sequentially process different views of the 3D environment. An example DNN may include a first stage that performs class segmentation in a first view (e.g., perspective view) and a second stage that performs class segmentation and/or regresses instance geometry in a second view (e.g., top-down). The DNN outputs may be processed to generate 2D and/or 3D bounding boxes and class labels for detected objects in the 3D environment. As such, the techniques described herein may be used to detect and classify animate objects and/or parts of an environment, and these detections and classifications may be provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.