Multi-View LiDAR Object Detection for Accurate 3D Box Orientation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional autonomous vehicle perception systems using Deep Neural Networks (DNNs) face challenges in accurately detecting and classifying objects, particularly pedestrians and bicycles, due to limitations in predicting 3D bounding boxes and object orientation from perspective and top-down views, leading to inefficiencies and reduced accuracy.
Innovation Solution
A multi-view perception system using DNNs that processes LiDAR data by sequentially segmenting and regressing object geometry in multiple views, including perspective and top-down views, to enhance object detection and classification, particularly for vulnerable road users like pedestrians and bicycles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional panoptic segmentation DNNs are used for perspective view object detection, then detection capability for certain classes like pedestrians and bicyclists is achieved, but accuracy in predicting 3D bounding boxes and object orientation is limited
Solution Approach 1:
The patent divides the object detection task into two separate segmentation stages: first performing panoptic segmentation in perspective view to identify objects and their classes, then performing a second segmentation in top-down view to accurately predict 3D bounding boxes and orientations. This multi-stage segmentation approach resolves the contradiction by improving measurement precision through additional processing while managing device complexity through modular architecture.
Solution Approach 2:
The patent transitions from two-dimensional perspective view detection to three-dimensional top-down view detection. By projecting detected objects from the perspective view into a top-down view and performing additional segmentation in this new dimensional space, the system achieves more accurate 3D bounding box predictions and orientation estimation, thereby improving measurement precision.
2Measurement precision
If conventional BEV object detection DNNs are used for top-down projection, then single class detection is achieved, but accuracy in detecting pedestrians and bicycles is limited due to similar top-down appearances
Solution Approach 1:
The patent applies sequential segmentation where the first segmentation in perspective view identifies object classes and locations, and the second segmentation in top-down view refines the detection by analyzing the overhead projection. This two-stage segmentation improves object classification accuracy by examining objects from multiple viewing angles, distinguishing between objects with similar top-down appearances through their perspective view characteristics.
Solution Approach 2:
The patent uses the perspective view detection results as an intermediary to guide the top-down view detection. By first identifying potential objects in the perspective view and then focusing the top-down analysis on these specific regions, the system improves classification accuracy while maintaining detection efficiency, avoiding the need to process the entire top-down view at full resolution.
3Measurement precision
If 3D convolutions are used over 3D voxelized volume for BEV detection, then detection accuracy is improved, but computational cost and processing time increase substantially
Solution Approach 1:
The patent segments the computational process into two distinct phases: perspective view processing using 2D convolutions for initial object identification, and top-down view processing using 2D convolutions on projected images for refined detection. This segmentation avoids the need for computationally expensive 3D convolutions over voxelized volumes while maintaining improved detection accuracy through multi-view analysis.
Solution Approach 2:
The patent creates a projected copy of the detected objects from the perspective view into the top-down view space. Instead of processing the entire 3D voxelized volume with 3D convolutions, the system generates 2D top-down projections of identified objects and performs efficient 2D convolutional processing on these copies, significantly reducing computational energy consumption while preserving detection accuracy.
Data Source
AI summary
A deep neural network(s) (DNN) may be used to detect objects from sensor data of a three dimensional (3D) environment. For example, a multi-view perception DNN may include multiple constituent DNNs or stages chained together that sequentially process different views of the 3D environment. An example DNN may include a first stage that performs class segmentation in a first view (e.g., perspective view) and a second stage that performs class segmentation and/or regresses instance geometry in a second view (e.g., top-down). The DNN outputs may be processed to generate 2D and/or 3D bounding boxes and class labels for detected objects in the 3D environment. As such, the techniques described herein may be used to detect and classify animate objects and/or parts of an environment, and these detections and classifications may be provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.