Activation Thresholds for Dynamic Neural Network Pruning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As artificial neural networks become more complex or the number of input data increases, the hardware resources required to output a final result also increase, leading to longer processing times.
Innovation Solution
A method for controlling the operation of a machine learning model by determining a threshold for skipping operations associated with activation values that do not affect the outcome, dynamically adapting the threshold based on activation value distributions, and expressing activation values as hardware-friendly floating point numbers for quicker determination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the artificial neural network is made more complex or the number of input data is increased, then the accuracy or capability of the model is improved, but the hardware resources required and processing time increase
Solution Approach 1:
The patent extracts and removes unnecessary operations from the neural network by identifying activation values below a threshold and skipping their associated computations. This extraction of redundant operations reduces hardware resource requirements and accelerates processing while preserving the model's essential functionality and accuracy.
Solution Approach 2:
The patent applies partial action by performing computations only for activation values that exceed the threshold, rather than processing all activation values uniformly. This selective computation approach reduces the total number of operations executed, thereby improving processing speed without completely sacrificing model accuracy.
2Reliability
If all operations are performed to ensure accurate results, then the reliability is improved, but the processing time increases
Solution Approach 1:
The patent implements skipping by allowing the system to bypass operations associated with activation values below the threshold. This selective skipping reduces processing time significantly while maintaining reliability for operations that truly impact the final result, as determined by the threshold-based filtering mechanism.
3Ease of operation
If a fixed threshold is used for skipping operations, then the ease of operation is improved, but the adaptability to different operating environments deteriorates
Solution Approach 1:
The patent transforms the static threshold into a dynamic parameter that adapts to different operating environments. The threshold is determined based on the distribution characteristics of activation values in the current environment, allowing the system to automatically adjust its operation skipping strategy to match the specific characteristics of each deployment scenario.
Solution Approach 2:
The patent changes the threshold parameter based on environmental conditions by analyzing the distribution of activation values. This parameter adaptation allows the system to optimize its performance for different operating environments, balancing between skipping operations and maintaining accuracy based on the actual data characteristics encountered.
Data Source
AI summary
A method for controlling operations of a machine learning model is performed by one or more processors and includes determining a threshold for skipping an operation, acquiring an activation value associated with at least one layer included in the machine learning model, determining whether the activation value is less than the threshold, and if the activation value is less than the threshold, controlling the operations of the machine learning model such that an operation associated with the activation value is skipped in the machine learning model.