Floating-Point Dot-Product Threshold Prediction for AI Energy Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The training stage of neural network models in artificial intelligence systems requires computationally intensive floating-point arithmetic computations, leading to significant energy consumption, particularly in performing dot-product operations.
Innovation Solution
A system comprising a floating-point computation unit and detection logic that computes the difference between the exponent parts of two floating-point values to predict whether the dot-product will be negative, thereby avoiding unnecessary computations and reducing energy usage by skipping a subset of operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If floating-point arithmetic computations are performed for training stage of neural network models, then computational accuracy is improved, but energy consumption increases significantly
Solution Approach 1:
The patent applies preliminary action by computing the sum of exponent parts before performing the complete floating-point dot-product operation. By evaluating the exponent sum in advance, the system can predict whether the final result will be negative or below a threshold, allowing early termination of computations when the outcome is predictable, thus reducing energy consumption while maintaining accuracy for cases that require full computation
Solution Approach 2:
The patent changes the parameter evaluation approach by separating the exponent part computation from the mantissa part computation. Instead of performing complete floating-point multiplication and addition, the system first computes only the exponent sum and uses it to determine whether further computation is necessary, effectively changing from a uniform computation approach to a conditional computation approach based on exponent analysis
2Measurement precision
If complete dot-product operations are performed in floating-point computation unit, then computational accuracy is maintained, but computational overhead increases
Solution Approach 1:
The detection logic performs a preliminary computation of the exponent sum before the complete dot-product operation. This preliminary action allows the system to identify cases where the final result will necessarily be negative or below threshold, enabling early termination and avoiding unnecessary computational overhead while preserving accuracy for cases that require complete computation
Solution Approach 2:
The patent applies partial action by performing only the necessary subset of computations. Instead of always executing the complete dot-product operation, the system performs only the exponent sum computation when sufficient to determine the outcome, and performs partial or complete mantissa computations only when the exponent analysis indicates the result may be above threshold, thus optimizing computational efficiency
3Use of energy by moving object
If threshold prediction is performed before dot-product computation, then energy consumption is reduced, but device complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the floating-point number representation into separate exponent and mantissa components, and by creating separate detection logic and floating-point computation unit modules. This segmentation allows the detection logic to independently evaluate exponent sums and make early termination decisions without requiring complex full-computation hardware, reducing overall energy consumption while managing device complexity through modular design
Data Source
AI summary
A system comprises a floating-point computation unit configured to perform a dot-product operation in accordance with a first floating-point value and a second floating-point value, and detection logic operatively coupled to the floating-point computation unit. The detection logic is configured to compute a difference between fixed-point summations of exponent parts of the first floating-point value and the second floating-point value and, based on the computed difference, detect the presence of a condition prior to completion of the dot-product operation by the floating-point computation unit. In response to detection of the presence of the condition, the detection logic is further configured to cause the floating-point computation unit to avoid performing a subset of computations otherwise performed as part of the dot-product operation. Such techniques serve as a predictor that the avoided subset of computations would have resulted in a dot product of the two floating-point values being below a threshold value.