Hierarchical Structured Sparsity for Faster Neural Network Pruning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional unstructured pruning of neural networks results in unpredictable zero distributions, leading to minimal performance gains or degradation, and requires customized hardware acceleration for varying sparsity degrees, making it challenging to construct efficient hardware support for a wide range of sparsity levels.
Innovation Solution
Implement hierarchical fine-grained structured sparsity, where each multi-element unit at the coarsest granularity is pruned to a desired sparsity degree, constraining the distribution of non-zero values within each unit, and associating hardware structures with per-level sparsity to enable efficient acceleration across various sparsity patterns and degrees.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If unstructured pruning is applied to achieve high model sparsity, then model size is reduced, but runtime performance degrades and hardware support becomes complex
Solution Approach 1:
The patent divides the weight tensor into multiple blocks along different dimensions (rows, columns, or channels), and applies structured sparsity at the block level rather than individual element level. This segmentation enables regular patterns of zeros that can be efficiently handled by hardware, while still achieving the desired overall sparsity degree for model size reduction.
Solution Approach 2:
The patent applies different sparsity patterns to different blocks within the weight tensor, allowing each block to have optimized local sparsity characteristics. This enables fine-grained control over the distribution of non-zero elements, balancing model compression with runtime performance requirements for specific hardware architectures.
2Quantity of substance
If unstructured sparsity of different degrees is applied, then model size is reduced, but hardware acceleration logic becomes drastically different and complex
Solution Approach 1:
The patent designs a unified hardware acceleration architecture that can handle multiple sparsity degrees (e.g., 50%, 66.7%, 75%, 80%) using the same basic computational units. The structured block-level sparsity patterns allow a single hardware design to efficiently support varying sparsity requirements without needing drastically different logic for each degree.
Solution Approach 2:
The patent changes the sparsity application level from individual elements to blocks, creating regular patterns that can be controlled by simple parameters (block size, sparsity ratio). This parameter-based control enables flexible sparsity degrees to be achieved through configuration rather than structural changes, simplifying hardware design.
3Productivity
If conventional structured sparsity is applied, then hardware acceleration is improved, but the variety of supported sparsity patterns and degrees is limited
Solution Approach 1:
The patent extends structured sparsity from traditional single-dimension patterns to multi-dimensional block structures. By organizing sparsity at the block level across multiple dimensions (spatial blocks, channel blocks, mixed blocks), the architecture supports a diverse variety of sparsity patterns while maintaining hardware acceleration efficiency through regular computational structures.
Data Source
AI summary
Hierarchical structured sparse parameter pruning and processing improves runtime performance and energy efficiency of neural networks. In contrast with conventional (non-structured) pruning which allows for any distribution of the non-zero values within a matrix that achieves the desired sparsity degree (e.g., 50%) and is consequently difficult to accelerate, structured hierarchical sparsity requires each multi-element unit at the coarsest granularity of the hierarchy to be pruned to the desired sparsity degree. The global desired sparsity degree is a function of the per-level sparsity degrees. Distribution of non-zero values within each multi-element unit is constrained according to the per-level sparsity degree at the particular level of the hierarchy. Each level of the hierarchy may be associated with a hardware (e.g., logic or circuit) structure that can be enabled or disabled according to the per-level sparsity. Hierarchical sparsity provides performance improvements for a greater variety of sparsity patterns, granularity, and sparsity degrees.