Pillar Pseudo-Images for Real-Time 3D Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in efficiently detecting 3D objects in complex environments due to the sparsity of sensor inputs from LIDAR point clouds, leading to computationally inefficient image-based deep learning techniques.
Innovation Solution
A computer system that processes 3D point clouds by dividing the space into vertical pillars, generating a sparse pseudo-image using 2D convolutions, and employing a neural network for object detection, reducing computational complexity and improving speed and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If traditional image-based deep learning techniques are used for 3D object detection, then the detection can be performed using standard 2D convolutional networks, but the computational efficiency is poor due to the sparsity of LIDAR point cloud data
Solution Approach 1:
The 3D space is segmented into multiple vertical pillars along the z-axis, with each pillar containing a subset of point cloud data. This segmentation transforms the sparse 3D point cloud into a structured format that can be efficiently processed by 2D convolutional networks, resolving the contradiction between ease of implementation and computational efficiency
Solution Approach 2:
The patent transforms the 3D point cloud data into a 2D pillar-based representation by organizing points along vertical pillars. This dimensionality reduction allows standard 2D CNNs to process LIDAR data efficiently without requiring complex 3D convolutional architectures, thereby improving computational efficiency while maintaining implementation simplicity
2Loss of information
If 3D point cloud data is processed directly using standard deep learning methods, then all 3D information is preserved, but the processing speed and accuracy are reduced due to data sparsity
Solution Approach 1:
By segmenting the point cloud into vertical pillars, the method organizes sparse 3D data into a denser 2D structure that preserves all original 3D information while enabling more effective feature extraction. This segmentation allows the network to process information more efficiently without losing any spatial data, thereby improving detection accuracy
Solution Approach 2:
The pillar-based representation serves as an intermediary structure between the raw 3D point cloud and the 2D convolutional network. This intermediate representation preserves all 3D information while transforming it into a format that 2D CNNs can process effectively, resolving the contradiction between information preservation and detection accuracy
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Among other things, we describe techniques for detecting objects in the environment surrounding a vehicle. A computer system is configured to receive a set of measurements from a sensor of a vehicle. The set of measurements includes a plurality of data points that represent a plurality of objects in a 3D space surrounding the vehicle. The system divides the 3D space into a plurality of pillars. The system then assigns each data point of the plurality of data points to a pillar in the plurality of pillars. The system generates a pseudo-image based on the plurality of pillars. The pseudo-image includes, for each pillar of the plurality of pillars, a corresponding feature representation of data points assigned to the pillar. The system detects the plurality of objects based on an analysis of the pseudo-image. The system then operates the vehicle based upon the detecting of the objects.