Bank-Balanced Sparsity for Neural Network Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional neural network model compression methods fail to achieve both high model accuracy and high hardware speedup, as fine-grained pruning introduces irregular computation and memory access, while coarse-grained pruning sacrifices model accuracy for hardware acceleration.
Innovation Solution
The bank-balanced sparsity method divides weight parameters into equal-sized banks and prunes them at the same sparsity level, allowing for fine-grained pruning within each bank to maintain accuracy while enabling efficient hardware implementation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If fine-grained pruning is applied to neural network weights, then model accuracy is maintained, but hardware speedup is limited due to irregular computation and memory access patterns
Solution Approach 1:
The patent divides the weight matrix into multiple banks of equal size, where each bank is pruned independently to the same sparsity level. This segmentation transforms the irregular fine-grained pruning into structured bank-level pruning, maintaining model accuracy through fine-grained control while enabling hardware acceleration through uniform computation patterns across banks
Solution Approach 2:
The patent applies different pruning strategies at different granularities: fine-grained pruning within each bank to preserve local weight importance and model accuracy, while applying coarse-grained uniform sparsity constraints across banks to enable hardware speedup. This local quality differentiation resolves the contradiction between accuracy and speed
2Productivity
If coarse-grained pruning is applied to neural network weights, then hardware speedup is achieved, but model accuracy is sacrificed
Solution Approach 1:
By segmenting weights into banks and pruning each bank independently at the same sparsity level rather than applying uniform coarse-grained pruning across all weights, the method maintains better model accuracy while still achieving hardware speedup through structured computation patterns
Solution Approach 2:
The patent changes the sparsity parameter application from a global coarse-grained approach to a local fine-grained approach within each bank, while maintaining uniform sparsity levels across banks. This parameter change enables both hardware speedup and accuracy preservation
Data Source
AI summary
In embodiments of the present disclosure, there is provided an approach for neural network model compression based on bank-balanced sparsity. In embodiments of the present disclosure, a set of weight parameters, such as a weight matrix, in a neural network is divided into a plurality of equal-sized banks in terms of number of elements, and then all of the equal-sized banks are pruned at the same sparsity level. In this way, each pruned bank will have the same number of non-zero elements, which is suitable for hardware speedup. Moreover, since each bank is pruned independently in a fine granularity, the model accuracy can be ensured. Thus, according to embodiments of the present disclosure, the neural network compression method based on bank-balanced sparsity can achieve both high model accuracy and high hardware speedup.