Sparse 3D Group Convolution for Computational Complexity Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current techniques for accelerating 3D convolutional neural networks (CNNs) focus on overall network architecture designs but lack efficient sparse 3D convolution methods, leading to computational and memory inefficiencies.
Innovation Solution
The proposed solution involves increasing the sparsity of input feature maps by partitioning them into multiple disjoint groups and applying shared 3D convolutional kernels to these groups, resulting in a more efficient sparse 3D group convolution approach.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional sparse 3D convolutions are used for 3D visual recognition tasks, then the network can process 3D data with depth dimension, but the computational complexity and memory requirements grow cubically
Solution Approach 1:
The patent partitions the input feature map into multiple disjoint groups along the depth dimension, allowing the convolution operation to be divided into smaller, independent group convolutions. This segmentation reduces the computational complexity from cubic O(D³) to linear O(D) with respect to the depth dimension, while maintaining the ability to process 3D visual data
2Adaptability or versatility
If conventional sparse 3D convolutions are used, then 3D visual recognition can be performed, but memory requirements increase due to the additional spatial dimension
Solution Approach 1:
By segmenting the feature map into G disjoint groups and applying shared convolutional kernels to each group, the patent reduces memory requirements. The shared kernels are applied across all groups, eliminating the need to store separate kernel weights for each depth position, thus reducing memory consumption while maintaining 3D processing capability
3Productivity
If the number of groups is increased in sparse 3D group convolution, then computational efficiency improves, but the complexity of the convolution operation increases
Solution Approach 1:
The patent employs shared convolutional kernels that are universally applied across all G disjoint groups. This universality allows the same kernel to perform multiple convolution operations on different groups, improving computational efficiency through reuse while keeping the actual convolution operation simple and manageable
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
An apparatus includes one or more processors (102) including a graphics processor (108) to process data; and a memory for storage of data, including feature maps. The one or more processors (102) are to provide for sparse 3D convolution acceleration by applying a shared 3D convolutional kernel/filter to an input feature map to produce an output feature map, including increasing sparsity of the input feature map by partitioning it into multiple disjoint input groups; generation of multiple disjoint output groups corresponding to the input groups by performing a convolution calculation represented by the shared 3D convolutional kernel/filter on all feature values associated with active/valid voxels of each input group to produce corresponding feature values within corresponding output groups; and outputting the output feature map by sequentially stacking the output groups.