Dynamic Sparsity Mode Selection in DNN Accelerators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current DNN accelerators face challenges in achieving efficient computation and energy usage due to fixed sparsity modes, which limit their ability to adapt to varying sparsity levels across different layers, leading to suboptimal performance and power consumption.
Innovation Solution
A dynamic sparsity-based acceleration approach that allows the DNN accelerator to switch between combined sparsity, weight sparsity, activation sparsity, and dense modes based on layer-specific configuration parameters and sparsity scores, optimizing energy consumption and performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If fixed sparsity modes are used in DNN accelerators, then device complexity is reduced, but adaptability to varying sparsity levels across different layers deteriorates
Solution Approach 1:
The patent implements dynamic sparsity mode selection where the accelerator can switch between combined sparsity mode, weight sparsity mode, activation sparsity mode, and dense mode based on layer-specific configuration parameters and sparsity scores. This dynamic adaptation resolves the contradiction by making the sparsity mode configurable per layer rather than fixed globally, allowing the system to balance complexity and adaptability.
2Use of energy by moving object
If combined sparsity mode is used for all layers, then energy consumption is reduced for sparse layers, but performance deteriorates for layers with low sparsity
Solution Approach 1:
The patent applies different sparsity modes to different layers based on their specific sparsity characteristics. Layers with high sparsity scores use combined sparsity mode for energy efficiency, while layers with low sparsity scores use dense mode or weight sparsity mode to maintain performance. This local optimization resolves the contradiction by tailoring the sparsity mode to each layer's needs rather than applying a uniform mode across all layers.
3Use of energy by moving object
If sparsity acceleration is applied uniformly across all layers, then overall energy consumption is reduced, but loss of information deteriorates due to inappropriate sparsity application
Solution Approach 1:
The patent changes the sparsity mode parameter dynamically based on layer-specific sparsity scores and configuration parameters. By adjusting the sparsity mode (combined, weight-only, activation-only, or dense) according to each layer's characteristics, the system minimizes energy consumption while preventing information loss that would occur from inappropriate sparsity application. The sparsity score calculation and mode selection mechanism ensures that sparsity is applied only where appropriate.
Data Source
AI summary
A deep neural network (DNN) accelerator may facilitate dynamic sparsity-based acceleration and operate in various sparsity modes including a combined sparsity mode, a weight sparsity mode, an activation sparsity mode, and a dense mode. The DNN accelerator may receive a configuration parameter indicating whether to accelerate the layer based on sparsity in a weight tensor of the layer. The configuration parameter may be generated offline, e.g., before the execution of the DNN is started. The DNN accelerator computes one or more activations of the layer in a previous layer in the DNN. The one or more activations are one or more elements of an activation tensor of the layer. The DNN accelerator may determine a sparsity mode for the layer based on the configuration parameter and sparsity in the activation tensor. One or more sparse cells in the DNN accelerator may execute the layer in the sparsity mode.