Neural Network Input Filtering to Cut IC Switching Power
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network accelerators consume excessive power due to complex computations, leading to performance throttling and increased latency, as they often perform numerous small-value calculations that can tolerate imprecision.
Innovation Solution
Implement a hardware mechanism with a filter circuit using a configurable bit mask to flush small values to zeros, reducing power consumption while maintaining computational accuracy by masking out certain values such as denormal numbers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If neural network accelerators perform complex computations to maintain high precision, then computational accuracy is improved, but power consumption increases
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the precision of arithmetic operations based on the input data characteristics. The system identifies when high precision is unnecessary and switches to lower precision modes, thereby reducing power consumption while maintaining adequate computational accuracy for the task at hand.
Solution Approach 2:
The patent implements partial action by selectively applying high precision computation only to the necessary portions of neural network operations. Rather than consistently using full precision across all computations, the system applies precision selectively based on data analysis, reducing overall power consumption while maintaining accuracy where needed.
2Productivity
If neural network accelerators operate at full capacity to maintain high performance, then productivity is improved, but heat generation increases
Solution Approach 1:
The patent applies dynamics by making the operational state of the accelerator flexible and adaptive rather than static. The system dynamically adjusts its computation intensity and precision levels based on real-time analysis of input data characteristics, allowing it to maintain high performance when needed while reducing heat generation during less critical operations.
Solution Approach 2:
The system changes operational parameters such as computation precision and activation thresholds based on input data analysis. By adjusting these parameters dynamically, the accelerator can operate at full capacity only when necessary, thereby maintaining productivity while reducing overall heat generation through selective high-performance operation.
3Measurement precision
If neural network accelerators perform numerous small-value calculations to maintain accuracy, then computational precision is improved, but switching activity increases
Solution Approach 1:
The patent applies parameter changes by adjusting the precision level of small-value calculations based on their actual impact on the final result. The system identifies small values that have negligible impact and performs computations on them with reduced precision or skips them entirely, thereby maintaining overall computational precision while significantly reducing switching activity and associated energy loss.
Data Source
AI summary
Techniques for replacing input values being loaded into a computational circuit are described. Small input values such as denormal numbers can be replaced with alternative values such as zeros to reduce switching activity in the computational circuit, and thus reduce power consumption. In applications such as most neural networks, the impact on the prediction results when replacing small numbers with zeros can be negligible. In applications where high precision computations may be desirable, the input values can be loaded into the computation circuit without modification.