Runtime Neural Network Predictor Suppresses Component Computations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks (ANNs) are computationally complex and energy-intensive due to the large data structures and immense number of discrete computations required, necessitating a reduction in computational complexity while maintaining fidelity to full execution.
Innovation Solution
The method involves generating a predictor at runtime using neural network and execution data to suppress less salient computations, reducing the number of computations required for ANN execution without the need for additional data structures or retraining, and optimizing data movement across computational nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If the number of computations is reduced by skipping or pruning, then computational complexity and energy consumption decrease, but the fidelity to full execution may be compromised
Solution Approach 1:
The patent applies runtime adaptability by dynamically determining which computations to suppress based on actual execution characteristics. The system generates a predictor at runtime using neural network data and execution data, then uses this predictor to dynamically suppress component computations. This dynamic approach allows the system to adapt to different input conditions and maintain fidelity while reducing computations, resolving the contradiction between energy consumption and execution fidelity.
Solution Approach 2:
The patent implements feedback mechanisms where execution data from the neural network is used to generate a predictor that guides computation suppression. The system continuously monitors execution characteristics and uses this feedback to adjust which computations are suppressed, ensuring that fidelity is maintained while reducing computational load. This feedback loop enables the system to learn from actual execution patterns and make informed decisions about computation suppression.
2Device complexity
If computations are suppressed based on static pruning or helper layers, then computational complexity decreases, but additional data structures or retraining are required
Solution Approach 1:
The patent applies self-service by having the neural network itself generate the predictor using its own execution data and neural network data. Instead of requiring external helper layers or separate pruning mechanisms, the system uses the neural network's own resources to determine which computations to suppress. This self-service approach reduces implementation complexity while maintaining computational efficiency.
Solution Approach 2:
The patent performs preliminary actions by generating the predictor at runtime before suppressing computations. The system first collects execution data and neural network data, then generates the predictor based on this data, and finally uses the predictor to guide computation suppression. This preliminary preparation ensures that the suppression decisions are informed by actual execution characteristics, reducing the need for complex pre-processing or retraining.
3Reliability
If all computations are executed, then fidelity to full execution is maintained, but computational complexity and data movement overhead increase
Solution Approach 1:
The patent applies local quality by selectively suppressing specific component computations based on their individual salience. Instead of uniformly reducing all computations, the system identifies and suppresses only those component computations that are less salient based on the predictor. This localized approach maintains fidelity in critical areas while reducing data movement overhead in less critical areas, resolving the contradiction between fidelity and data movement overhead.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods and systems relating to reducing the number of computations required to execute an artificial neural network (ANN) are disclosed herein. The methods include a computer-implemented method conducted during an execution of an ANN. The method includes generating a set of execution data, generating a summary of a set of neural network data of the ANN, generating a summary of a set of execution data of the execution of the ANN, generating a prediction using the summary of the set of neural network data and the summary of the set of execution data, and executing a composite computation. The composite computation is required for the execution of the ANN. The method also includes suppressing a set of component computations of the composite computation. The set of suppressed component computations is at least partly determined by the prediction.