Neural Network Weight Pruning for Lower Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of super large-scale weights in neural networks leads to increased storage and computational demands, resulting in higher memory access energy consumption and operation time, which current methods like low-rank matrix decomposition and hash techniques fail to adequately address.
Innovation Solution
A coarse-grained pruning method that selects weights based on a sliding window and sets them to zero when they meet a preset condition, followed by retraining using a back-propagation algorithm, and optionally quantizes weights to reduce precision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If low-rank matrix decomposition or hash techniques are used to reduce weights, then storage and computation are reduced to some extent, but the precision of the neural network is reduced and the reduction amount is limited
Solution Approach 1:
The patent extracts and removes redundant weights from the neural network through coarse-grained pruning. By using a sliding window to identify groups of weights and setting entire groups to zero when they meet pruning conditions, the method selectively removes unnecessary computational elements while preserving the network's essential functionality and precision.
Solution Approach 2:
The patent applies different treatment to different regions of the weight matrix through sliding window-based coarse-grained pruning. Each window region is evaluated independently, and pruning decisions are made locally based on the information quantity of weights within that specific region, allowing selective reduction without uniformly degrading network precision.
2Adaptability or versatility
If more weights are used in deeper and larger neural networks, then the network's representational capability is improved, but storage capacity requirements and memory access energy consumption increase
Solution Approach 1:
The patent extracts and eliminates redundant weights through coarse-grained pruning, removing unnecessary memory storage requirements and reducing memory access operations. By identifying and zeroing out entire groups of weights that contribute minimally to the network's representational capability, the method reduces storage capacity requirements and memory access energy consumption.
Solution Approach 2:
The patent changes the state of weight parameters from non-zero to zero through pruning operations. This parameter change reduces the effective number of weights that need to be stored and accessed, thereby reducing both storage capacity requirements and memory access energy consumption while maintaining adequate representational capability.
3Power
If more weights are used in neural networks, then the network's computational power is improved, but computing time and energy consumption increase
Solution Approach 1:
The patent extracts and removes redundant computational elements through coarse-grained pruning. By identifying weight groups with low information quantity and setting them to zero, the method reduces the number of computational operations required during forward and backward propagation, thereby reducing computing time and energy consumption while preserving essential computational power.
Solution Approach 2:
The patent applies computational optimization locally through sliding window-based pruning. Each window region is evaluated and pruned independently based on its contribution to the overall network performance, allowing selective reduction of computational operations in less critical regions while maintaining computational power in essential regions.
Data Source
AI summary
The present disclosure provides a processing device including: a coarse-grained pruning unit configured to perform coarse-grained pruning on a weight of a neural network to obtain a pruned weight, an operation unit configured to train the neural network according to the pruned weight. The coarse-grained pruning unit is specifically configured to select M weights from the weights of the neural network through a sliding window, and when the M weights meet a preset condition, all or part of the M weights may be set to 0. The processing device can reduce the memory access while reducing the amount of computation, thereby obtaining an acceleration ratio and reducing energy consumption.