Multi-View LiDAR Perception for Hidden-Feature Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional LiDAR perception systems for autonomous vehicles face challenges in accurately detecting and classifying objects, particularly those with distinguishing features not visible from the perspective view, leading to limited accuracy in predicting object classification, dimensions, and orientation, and are computationally inefficient due to 3D convolutions processing large volumes of empty voxel space.
Innovation Solution
A multi-view deep neural network (DNN) system that processes LiDAR data through multiple stages, first performing class segmentation in a perspective view and then regressing instance geometry in a top-down view, transforming classification data between views to enhance object detection accuracy and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If panoptic segmentation DNNs are used for perspective view object detection, then detection capability for objects with visible features is improved, but accuracy in predicting 3D bounding boxes and objects with hidden features deteriorates
Solution Approach 1:
The patent transforms 3D spatial information into 2D bird's eye view projections, allowing the network to process objects from multiple dimensional perspectives. This enables accurate prediction of 3D bounding boxes by combining 2D detection results with depth information from the transformed view, resolving the limitation of perspective view-only methods.
Solution Approach 2:
The patent segments the object detection task into multiple stages: first performing panoptic segmentation in perspective view to identify objects with visible features, then using bird's eye view transformation to detect objects with hidden features, and finally combining results to predict 3D bounding boxes. This multi-stage segmentation approach addresses both visible and hidden feature detection.
2Measurement precision
If 3D convolutions are used for BEV object detection, then detection accuracy is improved, but computational efficiency deteriorates due to processing large volumes of empty voxel space
Solution Approach 1:
The patent extracts only the necessary 2D projection information from 3D space, avoiding the need to process entire 3D voxel volumes. By taking out and processing only the relevant 2D bird's eye view features, the system achieves accurate object classification without the computational burden of full 3D convolution processing.
Solution Approach 2:
The patent changes from 3D voxel-based processing to 2D projection-based processing, significantly reducing the computational space from volumetric to planar. This dimensional transformation maintains detection accuracy while dramatically improving computational efficiency by eliminating processing of empty 3D space.
3Device complexity
If single-view DNNs are used for object detection, then device complexity is reduced, but detection accuracy for objects with hidden features deteriorates
Solution Approach 1:
The patent creates a multi-functional detection system where the bird's eye view transformation module serves multiple purposes: detecting objects with hidden features, providing 3D spatial context, and complementing perspective view detection. This universal approach improves overall detection accuracy without requiring entirely separate detection systems.
Solution Approach 2:
The patent introduces a bird's eye view transformation as an intermediary between perspective view input and final object detection output. This intermediary layer adds minimal complexity while enabling the network to access 3D spatial information and detect objects with hidden features that are invisible in the original perspective view.
Data Source
AI summary
A deep neural network(s) (DNN) may be used to detect objects from sensor data of a three dimensional (3D) environment. For example, a multi-view perception DNN may include multiple constituent DNNs or stages chained together that sequentially process different views of the 3D environment. An example DNN may include a first stage that performs class segmentation in a first view (e.g., perspective view) and a second stage that performs class segmentation and/or regresses instance geometry in a second view (e.g., top-down). The DNN outputs may be processed to generate 2D and/or 3D bounding boxes and class labels for detected objects in the 3D environment. As such, the techniques described herein may be used to detect and classify animate objects and/or parts of an environment, and these detections and classifications may be provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.