3D Point Cloud Neural Network Acceleration via Feature Separation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional 3D point cloud-based deep learning neural networks face challenges in accelerating 3D recognition due to high memory access and computation requirements, data sparsity, and duplicate computations, especially during sampling, grouping, and convolution operations.
Innovation Solution
A 3D point cloud-based deep learning neural network acceleration apparatus and method that limits the search area of a window-based 3D point search algorithm, shares hardware units for sampling and grouping, separates point and group feature data for efficient convolution computation, and applies bit-slice processing to skip sparse input and output computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If distance information between all 3D point data is calculated to uniformly sample and group adjacent 3D point data, then accurate 3D recognition is achieved, but memory access and computation requirements become excessively large
Solution Approach 1:
The patent divides the 3D point cloud into multiple local neighborhoods around each sampled point, calculating distances only within each local neighborhood rather than across the entire point cloud. This segmentation approach maintains accurate local 3D recognition while dramatically reducing the overall computation volume and memory access requirements.
Solution Approach 2:
The patent applies different processing strategies to different regions of the point cloud by computing distances only within local neighborhoods rather than uniformly across all points. This local quality approach allows accurate recognition in critical regions while skipping unnecessary computations in less critical areas.
2Measurement precision
If conventional ball query or KNN algorithms are used to group adjacent 3D point data, then accurate grouping is achieved, but huge amounts of memory access and computation are required
Solution Approach 1:
The patent segments the global grouping task into multiple local grouping tasks by computing distances only within local neighborhoods. This allows accurate grouping to be achieved at the local level while the overall computation efficiency improves due to the reduced search space compared to global ball query or KNN algorithms.
3Measurement precision
If convolution computation is performed on grouped 3D point data, then 3D recognition is achieved, but data sparsity and duplicate computations make acceleration difficult
Solution Approach 1:
The patent extracts and separates the point feature data from the group feature data in the convolution computation. By taking out the point features that remain constant across groups and processing them separately, the system eliminates duplicate computations while maintaining recognition accuracy, thereby simplifying the acceleration process.
Solution Approach 2:
The patent performs convolution computation only on the necessary group feature data after extracting and processing the point feature data separately. This partial action approach avoids the excessive computation that would occur if conventional convolution were applied to all grouped data, including redundant point features.
4Adaptability or versatility
If ReLU activation function is applied in convolution layers, then non-linearity is achieved, but input data sparsity causes 0 value computation that cannot be skipped
Solution Approach 1:
The patent extracts and processes only the non-zero elements of the input data after ReLU activation, separating them from the zero elements. This allows the system to maintain the non-linearity provided by ReLU while skipping the computation of zero values, thereby improving productivity and enabling computation skipping.
5Measurement precision
If max-pooling computation is performed on all output data in the last convolution layer, then 3D recognition is achieved, but unnecessary computation on output data occurs
Solution Approach 1:
The patent applies max-pooling computation only to the necessary output data elements that contribute to the final recognition result, rather than performing it on all output data. This partial action approach maintains the recognition accuracy while eliminating unnecessary computations, thereby improving computation efficiency.
Data Source
AI summary
Disclosed is a 3D point cloud-based deep learning neural network acceleration apparatus including a depth image input unit configured to receive a depth image, a depth data storage unit configured to store depth data derived from the depth image, a sampling unit configured to sample the depth image in units of a sampling window having a predetermined first size, a grouping unit configured to generate a grouping window having a predetermined second size and to group inner 3D point data by grouping window, and a convolution computation unit configured to separate point feature data and group feature data, among channel-direction data of 3D point data constituting the depth image, to perform convolution computation with respect to the point feature data and the group feature data, to sum the results of convolution computation by group grouped by the grouping unit, and to derive the final result.