Balanced-Weight Sparse Convolution Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hardware architectures are inefficient in storing and processing sparse convolution filters in neural networks, limiting computational efficiency and parallelism due to their reliance on general-purpose compute engines that are not optimized for sparse weights.
Innovation Solution
The method involves segmenting input tensors and filters into sub-tensors and sub-filters, respectively, with each sub-filter having a balanced number of non-zero weights, represented as index-value pairs for compact memory storage and parallel processing across multiple processors, allowing for efficient convolution operations and partial sum accumulation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If general-purpose compute engines (e.g., GPUs) are used for sparse convolution processing, then device availability and flexibility are maintained, but computational efficiency and parallelism are insufficient
Solution Approach 1:
The patent segments the convolution operation into multiple processing stages distributed across processor cores. Each core handles specific portions of the convolution computation, allowing parallel processing of sparse filters. This segmentation enables efficient utilization of available cores while maintaining flexibility in hardware configuration.
Solution Approach 2:
The patent introduces a new dimension of optimization by organizing sparse filter data in a specialized memory layout that exploits the sparsity pattern. By representing filters in a dimension optimized for sparse access patterns, the system achieves higher computational efficiency without requiring completely new hardware architecture.
2Quantity of substance
If sparse filters are stored in conventional memory formats, then implementation simplicity is maintained, but memory footprint and access efficiency are suboptimal
Solution Approach 1:
The patent extracts only the non-zero elements from sparse filters and stores them in a compact format. By removing zero elements entirely from the storage representation, the memory footprint is significantly reduced while preserving all necessary computational information for convolution operations.
Solution Approach 2:
The patent changes the representation parameters of filter data from dense floating-point arrays to a sparse format using index-value pairs. This parameter change optimizes both memory usage and access patterns for sparse convolution, achieving better efficiency despite increased format complexity.
3Productivity
If filters are segmented into sub-filters with balanced non-zero weights, then parallelism and load distribution are improved, but segmentation complexity increases
Solution Approach 1:
The patent segments filters into multiple sub-filters, each assigned to different processor cores. This segmentation creates balanced workloads across cores by distributing non-zero filter elements evenly, enabling efficient parallel processing while maintaining manageable complexity through systematic division.
Solution Approach 2:
The patent applies local quality optimization by ensuring each sub-filter assigned to a specific core has a balanced number of non-zero weights. This local balancing ensures uniform computational load across all cores, maximizing parallelism efficiency without requiring complex global optimization.
4Speed
If compact memory layout with index-value pairs is used, then memory access efficiency is improved, but processing overhead increases
Solution Approach 1:
The patent performs preliminary organization of sparse filter data into index-value pairs before convolution processing. This preliminary action pre-computes and stores the necessary indexing information, reducing the processing overhead during actual convolution by avoiding repeated sparsity pattern analysis.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for balanced-weight sparse convolution processing. An exemplary method comprises: obtaining an input tensor and a plurality of filters at a layer within a neural network; segmenting the input tensor into a plurality of sub-tensors; dividing a channel dimension of each of the plurality of filters into a plurality of channel groups; pruning each of the plurality of filters so that each of the plurality of channel groups of each filter comprises a same number of non-zero weights; segmenting each of the plurality of filters into a plurality of the sub-filters according to the plurality of channel groups; and assigning the plurality of sub-tensors and the plurality of sub-filters to a plurality of processors for parallel convolution processing.