Switchable Sparsity Acceleration in DNN Load Modules
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current DNN accelerators face challenges in achieving efficient computation and reducing area and power overhead due to fixed sparsity modes, which limit their ability to exploit both weight and activation sparsity effectively, leading to suboptimal performance and increased energy consumption.
Innovation Solution
A switchable sparsity acceleration approach that configures DNN accelerators to selectively skip MAC operations based on either weight sparsity or activation sparsity, allowing for dynamic selection between weight and activation sparsity modes to achieve greater compute acceleration while minimizing area and power overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If fixed sparsity mode is used in DNN accelerators, then device complexity is reduced, but compute acceleration and performance are limited
Solution Approach 1:
The patent implements a switchable sparsity mode mechanism that dynamically selects between weight sparsity mode and activation sparsity mode based on runtime conditions. The DNN accelerator includes a mode selection unit that can switch between different sparsity exploitation strategies, transforming the fixed architecture into a dynamic one that adapts to different computational scenarios, thereby achieving higher compute acceleration without proportionally increasing device complexity
Solution Approach 2:
The patent changes the operational parameter of the DNN accelerator by introducing switchable sparsity modes that alter how sparsity is exploited. By changing the sparsity mode parameter (weight sparsity vs. activation sparsity), the system can adapt to different input data characteristics and network layers, improving productivity without requiring complete architectural redesign
2Productivity
If two-sided sparsity acceleration is implemented, then compute acceleration is improved, but area and power overhead increase
Solution Approach 1:
The patent implements a dynamic sparsity mode selection mechanism that switches between weight sparsity mode and activation sparsity mode based on runtime conditions such as input data characteristics and network layer properties. This dynamic approach allows the system to achieve high compute acceleration when sparsity is beneficial while falling back to dense computation when it is not, thereby improving energy efficiency compared to always-enabled two-sided sparsity acceleration
Solution Approach 2:
The patent applies sparsity acceleration selectively rather than universally. The switchable mechanism enables partial application of sparsity techniques only when they provide genuine performance benefit, avoiding the excessive energy consumption that would result from applying two-sided sparsity acceleration in all scenarios, thus achieving a balance between compute acceleration and energy usage
Data Source
AI summary
A load module in a deep neural network (DNN) accelerator may receive a configuration parameter indicating a selection between an activation sparsity mode and a weight sparsity mode. The load module may read a sparse activation tensor, an activation sparsity bitmap, a sparse weight tensor, and a weight sparsity bitmap from a memory. The load module may densify one of the compressed tensors based on the sparsity mode and leave the other compressed tensor as is. The load module may load the dense tensor and the sparse tensor to a sparse cell. The sparse cell includes a sparsity module that may select one or more elements of the dense tensor based on the sparsity bitmap of the sparse tensor. The sparse cell also includes multiply-accumulate (MAC) units that perform MAC operation on the selected elements and the sparse tensor. MAC operations on unselected elements of the dense tensor are skipped.