Dynamic Neural Network Pruning for Inference Speed
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for reducing the parameters and complexity of Convolutional Neural Networks (CNNs) result in lower accuracy due to static pruning approaches that do not adapt to changing image contexts, making them inefficient for power-constrained devices like smart cameras.
Innovation Solution
Dynamic pruning of neurons on-the-fly using a lightweight branch path with a context aggregation component and fully connected layers for real-time importance classification, allowing for the exclusion of unimportant channels during neural network inferences, balancing accuracy and layer width loss constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If static pruning approaches are used to reduce the number of parameters in CNNs, then the computational cost is reduced, but the accuracy deteriorates due to inability to adapt to changing image contexts
Solution Approach 1:
The patent implements dynamic channel pruning that adapts to changing image contexts during inference. Instead of using a fixed pruning strategy, the system dynamically determines which channels to prune based on the actual image content and context, allowing the network to maintain accuracy while reducing computational cost for each specific inference task
Solution Approach 2:
The patent changes the pruning ratio parameter dynamically during inference based on the image context. The system adjusts the pruning ratio adaptively, allowing more aggressive pruning when image context allows and more conservative pruning when context requires higher accuracy, thus optimizing the balance between computational cost and accuracy for each scenario
2Productivity
If the number of channels is reduced to accelerate inference, then the processing time is reduced, but the layer width loss increases affecting accuracy
Solution Approach 1:
The system dynamically adjusts the number of active channels based on image context during inference. When the image context indicates that certain channels are less important, the system prunes those channels to accelerate inference. When context requires more channels for accurate processing, the system maintains more channels active, thus dynamically balancing inference speed and accuracy
Solution Approach 2:
The patent applies partial pruning rather than complete pruning, selectively removing only the least important channels based on context. This partial action approach allows the system to achieve speedup when possible while maintaining sufficient accuracy, avoiding the excessive action of removing too many channels that would harm performance
3Device complexity
If conventional pruning methods are used, then the network complexity is reduced, but the adaptability to different image contexts is lost
Solution Approach 1:
The patent implements a dynamic pruning mechanism that adapts to different image contexts during inference. The system analyzes the actual image content and adjusts the pruning strategy accordingly, maintaining adaptability to various imaging scenarios while reducing network complexity through context-aware channel selection and pruning
Solution Approach 2:
The system incorporates feedback mechanisms that monitor image context and adjust pruning decisions accordingly. This feedback loop ensures that the pruning strategy adapts to different imaging scenarios, maintaining the network's ability to handle diverse contexts while achieving reduced complexity through intelligent channel management
Data Source
AI summary
Systems, apparatuses and methods may provide for technology that aggregates contextual information from a first network layer in a neural network having a second network layer coupled to an output of the first network layer, wherein the context information is to be aggregated in real-time and after a training of the neural network, and wherein the context information is to include channel values. Additionally, the technology may conduct an importance classification of the aggregated context information and selectively exclude one or more channels in the first network layer from consideration by the second network layer based on the importance classification.