Deep Neural Network Weight Pruning for Edge Device Memory Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) require significant memory and computational resources, limiting their deployment on IoT platforms and edge devices without compromising classification accuracy.
Innovation Solution
The method involves reducing the weight between adjacent layers of a DNN using layer-independent ratios, local regular weight pruning, byte-aligned weight packing formats, and layer-by-layer fixed-point optimization, along with a simple and low-power decompressor hardware device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep neural network models are deployed on embedded platforms, then classification accuracy is improved, but memory usage and power consumption increase
Solution Approach 1:
The patent extracts and removes redundant or less important weights from the neural network model through pruning techniques. By identifying and eliminating unnecessary connections between neurons, the model maintains its classification accuracy while significantly reducing memory usage. This is achieved through iterative pruning processes that remove weights below certain thresholds while preserving the essential computational pathways.
Solution Approach 2:
The patent changes the precision parameters of weight representations by converting high-precision floating-point weights to lower-precision formats (such as INT8, INT4, or even binary). This parameter transformation reduces the memory footprint of the model while maintaining acceptable accuracy levels through quantization techniques that preserve the most significant weight characteristics.
2Measurement precision
If deep neural network models are deployed on embedded platforms, then classification accuracy is improved, but power consumption increases
Solution Approach 1:
By pruning redundant weights and connections from the neural network, the patent reduces the total number of computations required during inference. Fewer weight multiplications and additions directly translate to lower power consumption on embedded platforms, while the pruned model maintains its classification accuracy through careful selection of which weights to remove.
Solution Approach 2:
The patent changes the computational precision parameters from high-precision floating-point operations to lower-precision integer or binary operations. This parameter change reduces the energy required for each computational operation, as lower-precision arithmetic requires fewer transistor switching events and less activation energy, thereby reducing overall power consumption while preserving model accuracy.
3Quantity of substance
If weight precision is reduced to save memory, then memory usage decreases, but classification accuracy deteriorates
Solution Approach 1:
The patent applies different precision levels to different weights based on their importance rather than uniformly reducing precision across all weights. Critical weights that have greater impact on classification accuracy are preserved at higher precision, while less important weights are reduced to lower precision. This local differentiation of quality allows memory usage to be reduced while maintaining overall model accuracy.
Solution Approach 2:
The patent applies partial precision reduction only to specific portions of the model where it has minimal impact on accuracy. By selectively quantizing or pruning only certain layers or weight groups rather than the entire model, the patent achieves memory savings through partial action while preserving the accuracy-critical portions of the network at full precision.
4Quantity of substance
If aggressive pruning is applied to reduce model size, then memory usage decreases, but model complexity increases
Solution Approach 1:
The patent segments the pruning process into multiple iterative stages rather than applying aggressive pruning in a single step. Each stage removes a controlled portion of weights and allows for model evaluation and adjustment. This segmented approach prevents the model structure from becoming overly complex or unstable, as each pruning iteration can be carefully managed and reversed if necessary, while still achieving significant memory reduction.
Data Source
AI summary
A method for deep neural network compression is provided. The method includes: using at least one weight of a deep neural network (DNN), setting a value of a P parameter, and combining every P weights in groups, and perform branch pruning and retraining, so that only one of each group has a non-zero weight, and the remaining weights are 0, wherein the remaining weights are evenly divided into branches to adjust a compression rate of the DNN and to adjust a reduction rate of the DNN.