Neural Network Weight Compression for Low-Memory AI Acceleration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing scale and complexity of neural networks pose challenges in terms of storage capacity and energy consumption due to large-scale weights, which are difficult to accommodate in embedded devices and result in high access energy consumption.
Innovation Solution
A data compression method involving coarse-grained pruning and local quantization, including selecting weights based on a sliding window, retraining with back-propagation, grouping and clustering weights, and encoding center weights to form a codebook, reducing the need for storage and memory access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the scale of neural network is increased to improve processing capability, then the computational power and accuracy are improved, but the storage capacity requirement and energy consumption increase significantly
Solution Approach 1:
The patent extracts and removes redundant or less important weights from the neural network through pruning operations. By identifying and eliminating unnecessary connections between neurons, the network maintains its essential computational capabilities while reducing the total number of weights that need to be stored, thus resolving the contradiction between computational power and storage requirements.
Solution Approach 2:
The patent changes the precision parameter of weight storage by quantizing weights to lower bit representations. Instead of storing full-precision floating-point numbers, the system represents weights with reduced precision (e.g., 8-bit integers or even 1-bit binary values), which significantly reduces storage capacity requirements while maintaining acceptable computational accuracy for the neural network's productivity.
2Productivity
If the scale of neural network is increased to improve processing capability, then the computational power and accuracy are improved, but the energy consumption increases significantly
Solution Approach 1:
By pruning and removing unnecessary weights from the neural network, the patent reduces the total number of computational operations required during inference and training. Fewer weight accesses and computations directly translate to lower energy consumption while preserving the network's essential computational power for its intended tasks.
Solution Approach 2:
The quantization of weights to lower precision formats reduces the energy required for weight storage, retrieval, and processing. Lower-precision arithmetic operations consume less energy than full-precision operations, allowing the neural network to maintain computational power while significantly reducing the energy consumption associated with processing large-scale weights.
3Quantity of substance
If weight data is compressed to reduce storage size, then the storage capacity requirement is reduced, but the access energy consumption increases due to repeated access needs
Solution Approach 1:
The patent changes the precision parameter of weight representation to enable efficient compression. By quantizing weights to lower bit formats and using sparse representations where applicable, the system reduces storage size while optimizing the balance between compression ratio and access efficiency, thereby reducing both storage requirements and access energy consumption.
4Quantity of substance
If pruning is applied to reduce weight size, then the storage capacity requirement is reduced, but the manufacturing precision of weight configuration is affected
Solution Approach 1:
The patent systematically changes the precision parameter of weight representation through quantization, transforming full-precision floating-point weights into lower-precision formats. This controlled parameter change reduces weight size while managing the trade-off with precision, allowing the neural network to function effectively with compressed weight configurations.
Solution Approach 2:
The patent employs feedback mechanisms during the pruning and quantization processes to monitor and maintain network performance. By using validation sets and performance metrics, the system provides feedback to adjust pruning thresholds and quantization parameters, ensuring that weight configuration precision is maintained at acceptable levels while achieving significant compression.
Data Source
AI summary
The present disclosure provides a processing device including: a coarse-grained pruning unit configured to perform coarse-grained pruning on a weight of a neural network to obtain a pruned weight, an operation unit configured to train the neural network according to the pruned weight. The coarse-grained pruning unit is specifically configured to select M weights from the weights of the neural network through a sliding window, and when the M weights meet a preset condition, all or part of the M weights may be set to 0. The processing device can reduce the memory access while reducing the amount of computation, thereby obtaining an acceleration ratio and reducing energy consumption.