Lidar Point Cloud Pseudo Image Encoding for SoC Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep neural network-based 3D detection algorithms for LIDAR systems are computation-intensive, hardware unfriendly, and fail to meet real-time requirements, resulting in low detection accuracy and poor generalization.
Innovation Solution
The method involves converting irregular LIDAR points to a 3D grid, forming a pseudo image through reflection channel normalization, point decoration, and point feature encoding, allowing for high-dimensional feature extraction and prediction of 3D objects using a simplified neural network architecture optimized for System-on-Chip (SoC) hardware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If complicated point cloud encoding with 3D convolution and customized network structures is used, then detection accuracy is improved, but computational complexity increases and real-time requirements cannot be met
Solution Approach 1:
The patent creates a pseudo-image representation that copies the essential spatial and reflective information from the irregular point cloud into a regular 2D grid structure. This pseudo-image serves as a simplified copy that retains detection-relevant features while eliminating the computational burden of processing raw irregular points, thereby maintaining detection accuracy while reducing complexity
Solution Approach 2:
The patent transforms the data representation from 3D irregular coordinates to a 2D regular grid with normalized reflection channels. This parameter change in data structure allows the use of standard 2D convolution operations instead of complex 3D convolutions, significantly reducing computational complexity while preserving detection accuracy through the inclusion of reflection intensity information
2Device complexity
If point cloud is projected into BEV or FOV views, then computational complexity is reduced, but detection accuracy deteriorates
Solution Approach 1:
The patent projects the 3D point cloud onto a 2D plane while preserving depth information through the Z-coordinate encoding in the pseudo-image grid. This dimensional transformation maintains detection accuracy by keeping all three spatial dimensions represented in the 2D structure, unlike traditional BEV or FOV projections that lose depth information
Solution Approach 2:
The patent performs preliminary organization of point cloud data into a structured pseudo-image format before detection processing. This preliminary action assigns regular grid coordinates and normalizes reflection channels in advance, creating an optimized data structure that enables accurate detection with reduced computational complexity during the main detection process
3Reliability
If deep neural network-based algorithms are used, then detection capability is enhanced, but hardware friendliness decreases and real-time performance is unachievable
Solution Approach 1:
The patent replaces complex deep neural network architectures with a simplified detection algorithm that operates on the structured pseudo-image. This substitution uses standard 2D convolution operations and simple feature aggregation instead of complicated customized network structures, making the system hardware-friendly and suitable for deployment on resource-constrained chips while maintaining real-time detection capability
Data Source
AI summary
An example method of converting lidar points to a three-dimensional image, including receiving a set of irregular lidar points forming a lidar point cloud, assigning the set of irregular lidar points to a 3D or 2D grid resulting in a set of assigned points, determining a pseudo image based on the set of assigned points resulting in a set of regular pseudo image points, encoding the set of regular pseudo image points including a reflection channel normalization, at least one point decoration and a point feature of the at least one point decoration resulting in a set of high dimension regular features and predicting at least one three-dimensional object utilizing the set of high dimension regular features.