CNN Weight Pruning and Quantization for Mobile Deployment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning convolutional neural networks (CNNs) face challenges in deployment on resource-limited platforms due to large model size and high computational complexity, hindering their use on battery-powered mobile devices.
Innovation Solution
The method involves regularizing spatial domain weights, providing quantization, pruning small or zero weights, fine-tuning a quantization codebook, compressing the output, and utilizing sparse spatial domain or Winograd convolution to reduce memory and computational requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deep convolutional neural networks are deployed on resource-limited platforms, then computational capability and accuracy are improved, but model size and computational complexity increase memory and energy consumption beyond available resources
Solution Approach 1:
The patent segments the CNN model into multiple components that can be independently compressed: spatial domain weights, Winograd domain weights, and activation maps. Each component undergoes separate pruning and quantization operations, allowing selective compression without compromising overall model functionality. This segmentation enables the model to be divided into manageable parts that can be optimized for resource-constrained environments.
Solution Approach 2:
The patent applies parameter changes through quantization, transforming high-precision floating-point weights into lower-precision representations (e.g., 32-bit float to 8-bit integer or binary). This parameter transformation significantly reduces model size while maintaining acceptable accuracy. The method also changes the sparsity parameters by applying different pruning thresholds to different weight components, optimizing the balance between compression ratio and model performance.
2Reliability
If deep convolutional neural networks are deployed on resource-limited platforms, then computational capability is improved, but computational complexity increases energy consumption beyond available power
Solution Approach 1:
The patent extracts and removes redundant computational elements from the CNN model through pruning operations. By identifying and eliminating small or zero-valued weights in both spatial and Winograd domains, the method reduces the total number of multiply-accumulate operations required during inference. This extraction of unnecessary computations directly lowers energy consumption while preserving the essential computational capabilities needed for accurate predictions.
Solution Approach 2:
The patent applies partial action by performing pruning and quantization on only the most critical weight components rather than uniformly processing all parameters. The method selectively applies compression techniques to spatial domain weights and Winograd domain weights based on their contribution to model accuracy, avoiding excessive compression that would degrade performance. This selective approach optimizes the trade-off between energy savings and computational effectiveness.
3Quantity of substance
If model compression techniques are applied to reduce model size, then memory requirements are reduced, but model accuracy may deteriorate
Solution Approach 1:
The patent performs preliminary quantization and pruning operations during the model training and compression phase, before deployment on resource-constrained devices. By pre-computing quantized weight values and pruning masks, the method prepares an optimized model structure that maintains accuracy characteristics. The preliminary action includes computing sensitivity metrics and determining optimal pruning thresholds in advance, ensuring that compression does not significantly degrade model accuracy when deployed.
Solution Approach 2:
The patent incorporates feedback mechanisms through iterative pruning and quantization processes. After each compression operation, the model performance is evaluated, and pruning thresholds or quantization parameters are adjusted based on accuracy metrics. This feedback loop allows the method to identify the optimal compression level that maintains acceptable accuracy while achieving desired memory reduction targets. The feedback-driven approach ensures that compression operations do not push the model beyond its accuracy tolerance thresholds.
Data Source
AI summary
A method and system for constructing a convolutional neural network (CNN) model are herein disclosed. The method includes regularizing spatial domain weights, providing quantization of the spatial domain weights, pruning small or zero weights in a spatial domain, fine-tuning a quantization codebook, compressing a quantization output from the quantization codebook, and decompressing the spatial domain weights and using either sparse spatial domain convolution and sparse Winograd convolution after pruning Winograd-domain weights.