Multi-View LiDAR Perception for Accurate 3D Object Orientation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional autonomous vehicle perception systems using Deep Neural Networks (DNNs) face challenges in accurately detecting and classifying objects, particularly pedestrians and bicycles, due to limitations in predicting 3D bounding boxes and object orientation from perspective and top-down views, leading to inefficiencies and reduced accuracy.
Innovation Solution
A multi-view perception system using DNNs that processes LiDAR data by sequentially analyzing different views, including perspective and top-down views, to extract classification data and regress object geometry, enabling accurate detection and classification of objects like motorcycles, bikes, and pedestrians with high recall rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional panoptic segmentation DNNs are used for perspective view object detection, then detection speed is maintained, but accuracy in predicting 3D bounding boxes and object orientation deteriorates
Solution Approach 1:
The patent divides the object detection task into two separate processing streams: perspective view processing for detecting objects with distinguishing features, and top-down view processing for detecting objects without distinguishing features. Each stream uses specialized DNN architectures optimized for its specific task, allowing high accuracy in 3D bounding box prediction without requiring a single complex unified system.
Solution Approach 2:
The patent transforms the detection problem by projecting 3D LiDAR point clouds into both perspective views (preserving depth information for orientation estimation) and top-down views (providing overhead spatial context). This dimensional transformation allows the system to extract complementary geometric features from different viewing angles, improving 3D bounding box accuracy while maintaining computational efficiency.
2Measurement precision
If conventional BEV object detection DNNs are used for top-down projection, then single class detection is simplified, but accuracy in detecting pedestrians and bicycles deteriorates
Solution Approach 1:
The patent applies different processing strategies to different object classes based on their visual characteristics. For pedestrians and bicycles (objects without distinguishing top-down features), the system relies on perspective view information and 3D geometry. For other objects with clear top-down signatures, the system uses BEV processing. This localized optimization improves detection accuracy for vulnerable road users while maintaining overall system efficiency.
Solution Approach 2:
The patent introduces 3D bounding box predictions from the perspective view stream as an intermediary to guide and supplement the top-down view detection. The perspective-derived 3D geometry acts as a mediator that provides depth and orientation information missing from top-down views, enabling accurate detection of pedestrians and bicycles while maintaining efficient multi-class processing.
3Measurement precision
If 3D convolutions are used over 3D voxelized volume, then detection accuracy improves, but computational cost increases substantially
Solution Approach 1:
The patent extracts and utilizes the distinguishing geometric features from perspective views (such as height, orientation, and 3D shape) before processing, removing the need to process the entire 3D voxelized volume with computationally expensive 3D convolutions. By extracting key 3D characteristics early in the pipeline, the system achieves high detection accuracy while avoiding substantial computational overhead.
Solution Approach 2:
The patent performs preliminary 3D feature extraction and object hypothesis generation from perspective views before applying more intensive processing. This preliminary action identifies candidate objects and their approximate 3D parameters, allowing subsequent stages to focus computational resources only on refining detections for promising candidates rather than processing the entire 3D space uniformly.
Data Source
AI summary
A deep neural network(s) (DNN) may be used to detect objects from sensor data of a three dimensional (3D) environment. For example, a multi-view perception DNN may include multiple constituent DNNs or stages chained together that sequentially process different views of the 3D environment. An example DNN may include a first stage that performs class segmentation in a first view (e.g., perspective view) and a second stage that performs class segmentation and/or regresses instance geometry in a second view (e.g., top-down). The DNN outputs may be processed to generate 2D and/or 3D bounding boxes and class labels for detected objects in the 3D environment. As such, the techniques described herein may be used to detect and classify animate objects and/or parts of an environment, and these detections and classifications may be provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.