Dual-Sparse Convolution Parallelization With Rotating Filter Groups
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hardware architectures are inefficient in handling dual-sparse convolution scenarios due to their inability to efficiently store sparse tensors and exploit higher levels of parallelism, leading to increased computational and energy costs.
Innovation Solution
A method and system for parallelizing convolution processing by segmenting input tensors and filters into sub-tensors and sub-filter groups, assigning these to processors for parallel multiply-and-accumulate operations, and storing non-zero values in index-value pairs to reduce memory footprint and enhance efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If general-purpose compute engines (e.g., GPUs) are used for convolution processing, then versatility is maintained, but computational efficiency and energy efficiency deteriorate due to inability to efficiently store sparse tensors and exploit parallelism
Solution Approach 1:
The patent segments the convolution processing into multiple independent channel groups, where each group can be processed in parallel by separate compute units. This segmentation enables efficient exploitation of parallelism while maintaining the ability to handle different sparse patterns in each channel group, thus resolving the contradiction between versatility and processing efficiency.
Solution Approach 2:
The patent changes the data representation parameter by storing sparse tensors in a compressed format that records only non-zero elements and their positions. This parameter change enables efficient storage and processing of sparse data without requiring full-precision memory allocation, thereby improving computational efficiency while maintaining adaptability to different sparse convolution scenarios.
2Device complexity
If sparse convolution processing is implemented on general-purpose hardware, then implementation simplicity is maintained, but memory efficiency deteriorates due to inability to efficiently store sparse tensors
Solution Approach 1:
The patent changes the storage parameter from dense matrix format to sparse format, where only non-zero elements and their positions are stored. This parameter change dramatically reduces the memory footprint for sparse tensors while maintaining the ability to perform convolution operations, thus resolving the contradiction between implementation simplicity and memory efficiency.
3Device complexity
If dual-sparse convolution (both input tensors and filters are sparse) is processed using conventional methods, then algorithm simplicity is maintained, but computational cost and energy cost increase
Solution Approach 1:
The patent extracts and processes only the non-zero elements from both sparse input tensors and sparse filters, eliminating the need to perform multiplication operations on zero elements. This extraction approach significantly reduces the number of computational operations and associated energy consumption while maintaining algorithmic simplicity by only operating on relevant data elements.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for parallelizing convolution processing. An exemplary method comprises: segmenting an input tensor into a plurality of sub-tensors and a plurality of filters into a plurality of sub-filter groups; respectively assigning a plurality of combinations of the sub-tensors and the sub-filter groups to a plurality of processors; storing, by each of the plurality of processors, non-zero values of the sub-tensor and the sub-filter group in the assigned combination as index-value pairs; parallelly performing for a plurality of iterations, by the plurality of processors, multiply-and-accumulate (MAC) operations based on the index-value pairs to obtain a plurality of outputs, where the index-value pairs of the sub-filter groups are rotated among the plurality of processors across the plurality of iterations; and aggregating the plurality of outputs as an output tensor.