Runtime Predictors for ANN Computation Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks (ANNs) are computationally complex and energy-intensive due to the large data structures and immense number of discrete computations required, making them challenging to parallelize and optimize for reduced resource usage.
Innovation Solution
The method involves generating summaries of data to execute simplified composite computations, producing predictors that indicate salient computations, and suppressing less salient computations during execution, thereby reducing the overall computational complexity while maintaining fidelity to the full execution of the ANN.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the full execution of composite computations is performed in an ANN, then the accuracy and fidelity of the neural network output is maintained, but the number of computations and energy consumption increases significantly
Solution Approach 1:
The patent applies preliminary action by generating a predictor through simplified composite computations before executing the full composite computation. This predictor estimates which component computations are likely to be salient (have significant impact on output), allowing the system to pre-identify and suppress non-salient computations before they consume resources, thus reducing computation count while maintaining output fidelity.
Solution Approach 2:
The patent implements partial action by selectively executing only the salient component computations identified by the predictor, rather than executing all component computations. The predictor guides the execution to perform only the necessary partial set of computations that contribute significantly to the final output, thereby reducing the total number of computations while preserving the essential functionality and accuracy of the neural network.
2Reliability
If all component computations are executed to ensure accurate results, then the reliability of the computation is maintained, but the energy consumption and execution time increase
Solution Approach 1:
The system performs preliminary action by executing simplified composite computations to generate a predictor that forecasts which component computations will be salient. This preliminary step allows the system to identify and suppress non-salient computations before they consume energy, thereby reducing total energy consumption while maintaining computation accuracy through selective execution of only the necessary salient computations.
Solution Approach 2:
The patent applies the extraction principle by removing (suppressing) non-salient component computations from the execution pipeline based on predictor information. By extracting and eliminating the unnecessary computations that would not contribute significantly to the final output, the system reduces energy consumption and execution time while preserving the accuracy of the essential computations.
3Productivity
If the ANN structure is modified through pruning during training to reduce computations, then the number of computations is reduced, but the complexity of the training process increases
Solution Approach 1:
The patent applies preliminary action by generating a predictor through simplified composite computations before the main computation execution. This predictor serves as a guide to identify salient component computations, enabling the system to suppress non-salient computations during execution without requiring structural modifications to the ANN or complex pruning procedures during training, thus reducing computation count while avoiding increased training complexity.
Solution Approach 2:
The patent introduces an intermediary element - the predictor - which is generated by simplified composite computations and used to guide the execution of full composite computations. This intermediary mediator provides computation suppression guidance without requiring modification of the ANN structure or complexity of the training process, enabling computation reduction through runtime decision-making rather than structural changes.
4Ease of operation
If data is held in memory for all layers until computation to maintain readiness, then the availability of data for computations is ensured, but the memory usage and energy consumption increase
Solution Approach 1:
The patent applies the extraction principle by removing non-salient component computations from execution based on predictor information. Since these computations are identified as non-salient before execution, the data required for these suppressed computations does not need to be maintained in memory, thereby reducing memory usage and the associated energy consumption while preserving data availability for the essential salient computations.
Solution Approach 2:
The system performs partial action by executing only the salient component computations identified by the predictor. This selective execution means that data for non-salient computations can be discarded or not loaded into memory, reducing memory usage and energy consumption. The data availability is maintained for the partial set of salient computations that are actually executed, avoiding the need to hold all data in memory.
Data Source
AI summary
Methods and systems relating to reducing the number of computations required to execute an artificial neural network (ANN) are disclosed herein. A disclosed method includes: generating a summary of a set of data which is an input for a composite computation; executing a simplified composite computation, using the summary, to produce a simplified output; and executing a second simplified composite computation, using the simplified output, to produce a second simplified output which is a predictor. The second simplified composite computation is a simplification of a second composite computation. The composite computations are both part of a complex computation for the directed graph. The second composite computation depends on the composite computation in the directed graph. The method further includes suppressing, while executing the complex computation, a set of component computations from the second composite computation. The set of component computations are selected for suppression based on the predictor.