Hybrid Neural Network Pruning to Balance Accuracy and Model Size
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern neural network models, despite their utility, are not easily deployable on resource-constrained devices due to their large memory footprints and significant compute resource demands, making them unsuitable for devices with small form factors, limited memory, and modest processing capabilities.
Innovation Solution
A hybrid pruning approach combining coarse-grained channel pruning and fine-grained weight pruning is employed to reduce the size and computational complexity of neural networks, preserving accuracy by iteratively testing and refining the pruned models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If modern neural network models are deployed, then high accuracy and sophisticated processing capabilities are achieved, but large memory footprint and significant compute resource demands occur
Solution Approach 1:
The patent extracts and removes redundant or less important weights from the neural network model through pruning techniques. By identifying and eliminating unnecessary parameters, the model achieves reduced memory footprint while preserving the essential functionality and accuracy required for the application.
Solution Approach 2:
The patent applies different pruning strategies to different parts of the neural network based on their importance and characteristics. Critical layers and weights are preserved with higher fidelity, while less critical components are pruned more aggressively, creating a non-uniform sparsity pattern that optimizes the balance between accuracy and resource consumption.
2Measurement precision
If modern neural network models are deployed, then high accuracy and sophisticated processing capabilities are achieved, but significant compute resource demands occur
Solution Approach 1:
The patent removes computationally expensive operations and redundant calculations from the neural network through pruning. By eliminating unnecessary weights and simplifying the computational graph, the model requires fewer compute resources while maintaining its accuracy on resource-constrained devices.
Solution Approach 2:
The patent changes the parameters of the neural network by applying sparsity transformations and pruning ratios. These parameter modifications reduce the computational complexity of the model, enabling it to run efficiently on devices with limited processing power while preserving essential accuracy.
3Quantity of substance
If neural network pruning is applied, then model size is reduced for resource-constrained devices, but model accuracy may be degraded
Solution Approach 1:
The patent performs preliminary pruning actions on the neural network model before deployment, using training data and importance metrics to identify which weights can be safely removed. This advance preparation allows the model to be optimized for size while preserving accuracy-critical parameters, achieving the desired balance before the model is deployed to resource-constrained devices.
Solution Approach 2:
The patent employs feedback mechanisms during the pruning process by evaluating model performance on validation data and adjusting the pruning intensity accordingly. If accuracy degradation is detected, the pruning process is adjusted to preserve more weights, creating a feedback loop that optimizes the trade-off between model size and accuracy.
Data Source
AI summary
A pruned version of a neural network is generated by determining pruned versions of each a plurality of layers of the network. The pruned version of each layer is determined by sorting a set of channels of the layer based on respective weight values of each channel in the set. A percentage of the set of channels are pruned based on the sorting to form a thinned version of the layer. Accuracy of a thinned version of the neural network is tested, where the thinned version of the neural network includes the thinned version of the layer. The thinned version of the layer is used to generate the pruned version of the layer based on the accuracy of the thinned version of the neural network exceeding a threshold accuracy value. A pruned version of the neural network is generated to include the pruned versions of the plurality of layers.