Unbiased Minimum Variance Pruning for Neural Network Tensor Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep neural network (DNN) pruning technologies face limitations in accelerating the update phase during training, particularly due to the lack of efficient methods to accelerate the computation of gradients, which hinders overall processing efficiency.
Innovation Solution
The implementation of a tensor-level pruning method based on a minimum variance unbiased estimate (MVUE) approach, which prunes neural gradients to accelerate the update phase by determining pruning probabilities for elements in tensors, allowing for N:M structured sparsity and optimizing computations across all training phases.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If DNN pruning is applied to reduce computation cost, then computational efficiency is improved, but the update phase during training cannot be accelerated due to lack of efficient gradient computation methods
Solution Approach 1:
The patent segments the DNN training process into distinct phases (forward propagation, backward propagation, update phase) and applies targeted pruning strategies to each. Specifically, it prunes gradients during backward propagation and applies structured sparsity to weight updates, allowing each phase to be optimized independently rather than treating the entire training process uniformly.
Solution Approach 2:
The patent changes the sparsity parameter from fixed to dynamic by introducing N:M structured sparsity ratios that can be adjusted at different training stages. It also modifies the pruning probability parameter based on gradient magnitude statistics, allowing the pruning intensity to adapt during training while maintaining acceleration benefits throughout the update phase.
2Speed
If existing pruning methods are used, then some computational speedup is achieved, but accuracy degradation occurs and performance is not significantly improved
Solution Approach 1:
The patent implements feedback mechanisms by monitoring validation accuracy during training and dynamically adjusting the pruning probability and sparsity ratio accordingly. When accuracy degradation is detected, the system reduces pruning intensity; when accuracy stabilizes, it increases sparsity, creating a closed-loop control system that maintains reliability while maximizing speedup.
Solution Approach 2:
The patent applies partial pruning strategically by identifying and preserving critical gradients and weights while pruning less important connections. Rather than uniformly pruning all parameters, it uses magnitude-based selection to prune only a portion of connections in each layer, maintaining sufficient computational pathways for accurate inference while achieving speedup.
3Ease of manufacture
If uniform pruning is applied to all parameters, then implementation is simple, but computational resources are not optimized effectively
Solution Approach 1:
The patent applies local quality by differentiating pruning strategies across different parameter types (gradients vs. weights) and different layers of the network. It uses magnitude-based local selection within each parameter group, applying higher pruning probabilities to parameters with smaller magnitudes while preserving larger-magnitude parameters, thereby optimizing resource allocation locally rather than uniformly across the entire network.
Data Source
AI summary
A DNN can be compressed by pruning one or more tensors for a deep learning operation. A first pruning parameter and a second pruning parameter are determined for a tensor. A vector having a size of the second pruning parameter may be extracted from the tensor. Pruning probabilities may be determined for the elements in the vector. One or more elements in the vector are selected based on the pruning probabilities. Alternatively, a matrix, in lieu of the vector, may be extracted from the tensor. Pruning probabilities may be determined for the columns in the matrix. One or more columns are selected based on their pruning probabilities. The number of the selected element(s) or column(s) may equal the first pruning parameter. The tensor can be modified by modifying the value(s) of the selected element(s) or column(s) and setting the value(s) of one or more unselected elements or columns to zero.