Balanced Pruning for Deep Neural Network Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Hardware accelerators for deep neural networks face inefficiencies in processing sparse parameters, as balancing sparsity across parallel processing elements is challenging, affecting computing efficiency and accuracy.
Innovation Solution
A system and method for balanced pruning of deep neural network weights, using a weight masker and loss determiner to partition weights into groups, apply a masking function, determine non-zero weights, and minimize loss with a Lagrange multiplier to enforce equal variance of non-zero weights across groups, followed by post-processing to achieve exact sparsity balance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If sparsity is increased to reduce parameter size and increase computing efficiency, then computing efficiency is improved, but balancing sparsity across parallel processing elements becomes difficult and workloads become unbalanced
Solution Approach 1:
The patent transforms the discrete, non-differentiable sparsity pruning problem into a continuous, differentiable optimization problem by introducing a soft masking function with trainable parameters. This allows sparsity to be controlled as a continuous parameter during training, enabling gradient-based optimization while maintaining the ability to balance workloads across processing elements through the loss function's variance term.
Solution Approach 2:
The patent implements feedback by incorporating a variance term in the loss function that monitors and adjusts the distribution of non-zero weights across groups. The Lagrange multiplier dynamically adjusts the sparsity constraint strength based on the observed variance, providing continuous feedback to balance workloads across parallel processing elements while maintaining computing efficiency.
2Quantity of substance
If weights are pruned to reduce parameter size, then parameter size is reduced, but the distribution of non-zero weights becomes unbalanced across groups
Solution Approach 1:
The patent uses a soft masking function with trainable parameters to control sparsity, allowing continuous adjustment of the proportion of non-zero weights in each group. This parameter-based control enables simultaneous reduction of overall parameter size while maintaining uniform distribution across groups, resolving the contradiction between parameter size reduction and distribution uniformity.
Solution Approach 2:
The variance term in the loss function provides continuous feedback on the distribution uniformity of non-zero weights across groups. During training, this feedback mechanism adjusts the masking parameters to minimize variance while maintaining the desired sparsity level, ensuring both parameter size reduction and uniform distribution are achieved.
3Quantity of substance
If traditional pruning methods are used to reduce parameters, then parameter size is reduced, but hardware accelerator efficiency is not optimized due to unbalanced workloads
Solution Approach 1:
The patent transforms traditional discrete pruning into a continuous parameter optimization problem using soft masking functions. This allows the sparsity pattern to be controlled by trainable parameters that are optimized during training to balance workloads across hardware accelerators, thereby improving hardware efficiency while maintaining parameter size reduction.
Solution Approach 2:
The loss function incorporates a variance term that provides feedback on workload balance across hardware processing elements. This feedback mechanism guides the optimization of masking parameters to achieve balanced sparsity distribution, directly improving hardware accelerator efficiency while maintaining reduced parameter size.
Data Source
AI summary
A system and method that provides balanced pruning of weights of a deep neural network (DNN) in which weights of the DNN are partitioned into a plurality of groups, a count of a number of non-zero weights is determined in each group, a variance of the count of weights in each group is determined, a loss function of the DNN is minimized using Lagrange multipliers with a constraint that the variance of the count of weights in each group is equal to 0, and the weights and the Lagrange multipliers are retrained by back-propagation.