Neural Network Pruning Markers for Low-Power Edge Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing demand for higher capabilities and accuracy in neural network inference services leads to increased data size and computational power requirements, straining the performance of processors and memory in low-power neural processing units at edge devices.
Innovation Solution
Implement pruning markers to track the pruning of weight parameters in neural network models, iteratively reducing the size of weight parameters while maintaining a loss function within a threshold, and converting functions into graph modules for efficient operation on low-power neural processing units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the number of hidden layers and model complexity are increased to improve inference accuracy, then model capability and accuracy are improved, but computational power requirements and processor performance demands increase
Solution Approach 1:
The patent extracts and removes redundant or less important weight parameters from the neural network model through pruning operations. By identifying and eliminating unnecessary connections between neurons, the model maintains its inference accuracy while reducing the total number of parameters that require computational processing, thereby lowering power requirements.
Solution Approach 2:
The patent applies different pruning strategies to different parts of the neural network based on the importance of individual weight parameters. By analyzing the magnitude and contribution of each parameter locally, the system preserves critical parameters while removing less important ones, achieving a non-uniform optimization that maintains accuracy while reducing computational load in specific regions of the model.
2Use of energy by moving object
If the number of weight parameters is reduced through pruning to decrease computational demands, then power consumption is reduced, but model accuracy may deteriorate
Solution Approach 1:
The patent implements a feedback mechanism where the pruning process is guided by monitoring the loss function during iterative pruning operations. The system continuously evaluates whether removing weight parameters causes the loss to exceed a predefined threshold, and if so, it stops pruning or adjusts the pruning strategy. This feedback loop ensures that power consumption is reduced through parameter removal while model accuracy is preserved by preventing excessive pruning that would degrade performance.
Solution Approach 2:
The patent applies partial pruning by removing only a portion of the weight parameters rather than eliminating them entirely. By carefully controlling the extent of pruning to stay within acceptable accuracy boundaries (using loss function thresholds), the system achieves energy reduction through parameter removal without excessive action that would harm model performance.
3Quantity of substance
If iterative pruning is applied to reduce weight parameter size while maintaining loss function threshold, then model size is reduced for low-power operation, but processing time and complexity of the pruning process increase
Solution Approach 1:
The patent performs preliminary analysis of weight parameter importance before the actual pruning execution. By pre-calculating metrics such as parameter magnitudes and their contributions to the overall model output, the system prepares a prioritized list of parameters for removal. This preliminary action reduces the computational burden during the iterative pruning process itself, as the system can quickly identify candidate parameters for removal without extensive re-evaluation at each step.
Data Source
AI summary
A method may comprise: converting one or more functions or function call instructions of a first neural network (NN) model into one or more graph modules; analyzing a relationship between one or more inputs and one or more outputs of the one or more graph modules; generating a second NN model including the one or more graph modules as one or more nodes of a directed acyclic graph (DAG) by coupling the one or more inputs and outputs of the graph modules based on the relationship; adding one or more markers corresponding to a weight parameter of one or more layers of the second NN model; and updating the one or more markers according to a pruning algorithm that removes at least a portion of the weight parameter.