Neural Network Model Compression via Layer-Wise Pruning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI technologies face challenges in deploying Deep Neural Networks (DNNs) on resource-constrained devices like smartphones and drones due to high computational costs and memory requirements, which lead to rapid battery discharge and inefficiency.
Innovation Solution
A method and system for compressing neural network models by determining filter and position-wise contribution information, layer scores, and pruning control parameters to set a layer-wise pruning rate, allowing for efficient compression without compromising accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If DNN models are deployed on portable devices, then AI functionality is provided, but computational cost and energy consumption increase
Solution Approach 1:
The patent extracts and removes redundant or less important filters from the neural network model based on contribution analysis. By identifying and eliminating unnecessary computational elements (filters with low contribution scores), the model size and computational requirements are reduced while maintaining essential AI functionality on portable devices
Solution Approach 2:
The patent applies different pruning ratios to different layers of the neural network based on their specific contribution to the output. Instead of uniform compression, each layer is analyzed and pruned locally according to its importance metrics, allowing critical layers to retain more filters while less critical layers are compressed more aggressively
2Adaptability or versatility
If DNN models are deployed on portable devices, then AI functionality is provided, but memory requirements increase
Solution Approach 1:
The patent removes redundant filters and their associated weight parameters from the model. By extracting only the essential computational elements needed for accurate inference, the overall model size and memory footprint are reduced, enabling deployment on devices with limited storage capacity
Solution Approach 2:
The patent changes the parameter configuration of the neural network by adjusting the number of filters in each layer according to contribution analysis. This parameter optimization reduces the total number of weights and biases that need to be stored in memory while preserving the model's predictive capability
3Adaptability or versatility
If DNN models are deployed on portable devices, then AI functionality is provided, but processing capacity requirements increase
Solution Approach 1:
The patent extracts and eliminates computationally expensive but less important filter operations from the model. By removing these redundant computational tasks, the overall processing capacity required for inference is reduced, making the model suitable for execution on portable devices with limited CPU/GPU power
Solution Approach 2:
The patent applies differentiated pruning strategies to different layers based on their computational cost and contribution. Layers with high computational cost but low contribution are pruned more aggressively, while layers with high contribution are preserved, optimizing the overall processing requirements
4Productivity
If model compression is applied, then computational cost is reduced, but model accuracy may deteriorate
Solution Approach 1:
The patent uses contribution analysis to dynamically adjust the number of filters in each layer based on their measured importance to the model output. By changing the filter count parameter proportionally to contribution scores, the compression maintains the essential computational pathways needed for accurate inference while removing redundant capacity
Solution Approach 2:
The patent applies different compression ratios to different layers based on their local contribution to the final output. Critical layers maintain higher filter counts to preserve accuracy, while less critical layers are compressed more aggressively, achieving overall model compression without significant accuracy loss
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method and system of compressing a neural network model (NNM) is disclosed. The method includes determining filter contribution information and position wise contribution information of each of the plurality of layers based on a total number of the plurality of layers in the NNM, a total number of the plurality of filters in the NNM, and a number of filters in each of the plurality of layers. A layer score is determined based on a type of layer for each of the plurality of layers and a predefined scoring criteria. A pruning control parameter is determined of each of the plurality of layers based on the layer score, the filter contribution information and the position wise contribution information of the corresponding layers. A layer-wise pruning rate is determined of each of the plurality of layers based on the pruning control parameter and the pre-defined pruning ratio.