Dynamic Neural Network Channel Pruning for Accuracy-Preserving Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep neural networks (DNNs) face challenges in reducing computation and storage requirements while maintaining accuracy, particularly on mobile devices with limited resources, due to redundancy in parameters and complexity in channel pruning methods.
Innovation Solution
A method for dynamically adjusting neural network channels by selecting initial pruned and unpruned channels based on a preset pruning rule, training the network, recovering incorrectly pruned channels, and iteratively adjusting channel ratios until a target ratio is reached, ensuring minimal accuracy loss.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If channel pruning is performed to reduce model size and computation, then storage requirements and processing complexity are reduced, but network accuracy decreases
Solution Approach 1:
The patent implements dynamic channel pruning by iteratively adjusting the pruning ratio based on network performance feedback. The system dynamically determines which channels to prune and which to retain during training, allowing the model to adaptively reduce complexity while maintaining accuracy through continuous monitoring and adjustment of pruning parameters.
Solution Approach 2:
The patent changes the pruning ratio parameter dynamically during training. By adjusting the pruning ratio from 0% to target values (e.g., 50%, 70%, 90%) in different training stages, the system optimizes the balance between model size reduction and accuracy maintenance, allowing progressive compression without catastrophic performance loss.
2Quantity of substance
If existing channel pruning algorithms are used, then model compression is achieved, but training complexity and computational cost increase significantly
Solution Approach 1:
The patent performs preliminary channel importance assessment during initial training by computing channel-wise statistics (mean, variance, max absolute value) before applying pruning. This preliminary analysis identifies candidate channels for pruning without requiring complex iterative optimization, significantly reducing training complexity while achieving effective compression.
Solution Approach 2:
The patent extracts and removes only the least important channels based on pre-computed metrics, rather than using complex global optimization methods. By directly eliminating channels with lowest importance scores identified through statistical analysis, the system achieves compression with minimal training complexity and without requiring sophisticated pruning algorithms.
3Reliability
If more stacked layers are added to improve DNN performance, then prediction accuracy improves, but storage requirements and processing complexity increase
Solution Approach 1:
The patent extracts and removes redundant channels from deep network layers, identifying and eliminating channels that provide minimal contribution to prediction accuracy. This selective extraction allows the network to maintain its depth and representational capacity while removing unnecessary parameters, thereby preserving accuracy with reduced model size.
Solution Approach 2:
The patent applies different pruning strategies to different channels within the same layer based on their individual importance metrics. By evaluating and treating each channel according to its local contribution to network performance, the system maintains critical representations while removing redundant ones, preserving overall network accuracy despite reduced parameter quantity.
Data Source
AI summary
Disclosed are a method, apparatus and device for dynamically adjusting a neural network channel, and a computer-readable storage medium. The method comprises: selecting, on the basis of a preset pruning rule, initial pruned channels and initial unpruned channels of an initial deep neural network; after the initial deep neural network is trained, correcting a channel that is incorrectly pruned, and dynamically adjusting the number of network channels; cyclically executing the steps of training a network, correcting channel pruning, and determining whether the ratio of the number of current pruned channels to the total number of channels reaches a preset proportion threshold value, until the ratio of the number of current pruned channels to the total number of channels reaches the preset proportion threshold value; and obtaining a deep neural network after network channel pruning is completed, and training the deep neural network to obtain a target deep neural network. By means of the method, the apparatus, the device, and the computer-readable storage medium provided in the present invention, the network computation amount and the model size are reduced, but the network precision is not reduced.