Dynamic Neural Network Surgery for Mobile AI Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep neural networks (DNNs) face challenges in efficient implementation on mobile devices due to high computation and memory requirements, and current compression techniques often result in accuracy loss.
Innovation Solution
The method involves pruning and splicing operations on a trained DNN to generate a sparse neural network model, which reduces the number of connections and updates weights iteratively, allowing for dynamic network surgery and maintaining high accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If deep neural networks are deployed on mobile devices, then artificial intelligence functionality is achieved, but computation and memory requirements become prohibitively heavy
Solution Approach 1:
The patent extracts and removes redundant connections (weights) from the deep neural network through iterative pruning operations. By identifying and eliminating unnecessary connections between neurons across layers, the model size and memory requirements are significantly reduced while preserving essential functionality for mobile deployment
Solution Approach 2:
The patent applies different treatment to different connections within the network. Instead of uniform compression, it selectively prunes specific connections based on their importance metrics (such as weight magnitude or contribution to output), maintaining high-quality connections while removing low-quality ones, thereby optimizing the local structure of each layer for mobile efficiency
2Adaptability or versatility
If deep neural networks are deployed on mobile devices, then artificial intelligence functionality is achieved, but computation requirements become prohibitively heavy
Solution Approach 1:
The patent removes redundant computational paths by pruning unnecessary connections between neurons. This extraction of essential computation from the full network reduces the total number of operations required during inference, making the model computationally feasible for mobile devices with limited processing power
Solution Approach 2:
The patent segments the network into important and unimportant connections through iterative pruning. By dividing the computational graph and selectively retaining only essential connections, the computation is segmented into necessary operations only, reducing overall computational burden while maintaining accuracy
3Quantity of substance
If lossy compression techniques are applied to DNN models, then model size is reduced, but accuracy is lost
Solution Approach 1:
The patent changes the parameters of connections (weights) through iterative pruning and retraining. By dynamically adjusting which connections are retained and retraining the pruned network, the model adapts to maintain accuracy while achieving significant compression. The parameter changes are guided by importance metrics that preserve critical information
4Manufacturing precision
If lossless compression techniques are applied to DNN models, then accuracy is maintained, but training efficiency is reduced due to multiple iterations
Solution Approach 1:
The patent performs preliminary pruning actions based on pre-computed importance metrics before full retraining. By预先 identifying and removing obviously redundant connections using efficient metrics (such as weight magnitude or gradient-based importance), the subsequent retraining process starts from a better initial state, reducing the total number of iterations needed compared to traditional alternating prune-retrain cycles
Data Source
AI summary
Techniques related to compressing a pre-trained dense deep neural network to a sparsely connected deep neural network for efficient implementation are discussed. Such techniques may include iteratively pruning and splicing available connections between adjacent layers of the deep neural network and updating weights corresponding to both currently disconnected and currently connected connections between the adjacent layers.