Neural Network Weight Compression for Low-Memory AI Acceleration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing scale and complexity of neural networks lead to challenges in storage capacity and energy consumption due to large-scale weights, which are difficult to store and access efficiently, particularly in resource-constrained devices like mobile phones.
Innovation Solution
A data compression method involving coarse-grained pruning and local quantization of neural network weights, including selecting weights based on a sliding window, setting them to zero, retraining, grouping, clustering, and encoding center weights to form a codebook, while only training the codebook during retraining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If large-scale weight data is used in deep neural networks, then model accuracy and capability are improved, but storage capacity requirements increase significantly
Solution Approach 1:
The patent extracts and removes redundant weight parameters from the neural network through pruning operations. By identifying and eliminating weights that contribute minimally to model performance, the system reduces the total quantity of weight data while preserving the essential functionality and accuracy of the network.
Solution Approach 2:
The patent applies quantization to change the precision parameters of weight data from high-precision floating-point representations to lower-precision formats. This parameter transformation significantly reduces the storage capacity required for weight data while maintaining acceptable model accuracy through careful quantization strategies.
2Reliability
If large-scale weight data is stored and accessed multiple times, then model computation capability is maintained, but energy consumption increases significantly
Solution Approach 1:
By pruning redundant weights, the system reduces the total number of weight parameters that need to be loaded and accessed during computation. This extraction of unnecessary data directly reduces the energy consumption associated with memory access operations while maintaining the computational capability required for accurate predictions.
Solution Approach 2:
Quantization changes the bit-width parameters of weight data, enabling more efficient storage and faster access times. The reduced precision requirements allow for more compact weight representations that can be loaded into cache memory more efficiently, reducing the frequency and energy cost of memory access operations.
3Quantity of substance
If weight data is compressed through pruning and quantization, then storage overhead is reduced, but computation precision may be affected
Solution Approach 1:
The patent applies different compression strategies to different parts of the weight matrix based on their importance. Pruning operations selectively remove only the least important weights, while quantization applies different precision levels to different weight groups. This local differentiation maintains computation precision for critical weights while achieving compression for less critical parameters.
Solution Approach 2:
The system applies pruning and quantization partially rather than universally to all weight parameters. By selectively applying compression techniques only where they have minimal impact on accuracy, the system achieves storage reduction while preserving computation precision for the most important weight parameters.
Data Source
AI summary
The present disclosure provides a processing device including: a coarse-grained pruning unit configured to perform coarse-grained pruning on a weight of a neural network to obtain a pruned weight, an operation unit configured to train the neural network according to the pruned weight. The coarse-grained pruning unit is specifically configured to select M weights from the weights of the neural network through a sliding window, and when the M weights meet a preset condition, all or part of the M weights may be set to 0. The processing device can reduce the memory access while reducing the amount of computation, thereby obtaining an acceleration ratio and reducing energy consumption.