Floating-Point Unit Dot-Product Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing methods for computing floating-point dot-products in artificial neural networks are computationally intensive due to the need to determine the maximum exponent and right-shift mantissa products, which introduces latency and slows down the computation process, especially in large-scale neural network operations.
Innovation Solution
The proposed solution involves dividing the product of exponents into high and low portions, allowing for early right-shifting of mantissa products and reducing the latency by calculating the maximum exponent only on the high portion, enabling faster computation of dot-products in two cycles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the maximum exponent is determined by comparing all N product values of exponents, then the exponent alignment is accurate, but the computation time increases due to requiring several processing cycles
Solution Approach 1:
The patent segments the M-bit exponent comparison into two stages: first comparing the most significant K bits to identify candidate maximum exponents, then comparing the remaining least significant (M-K) bits among candidates. This segmentation reduces the total number of comparisons needed while maintaining full precision accuracy.
Solution Approach 2:
The patent performs preliminary comparison of the most significant K bits before completing the full exponent comparison. By identifying candidate maximum exponents based on MSB comparison first, the system prepares the groundwork for the final precise comparison, reducing the computational burden of the complete operation.
2Measurement precision
If the full M-bit exponent comparison is performed before right-shifting mantissas, then the alignment is precise, but the latency increases due to sequential processing
Solution Approach 1:
The patent performs preliminary right-shifting of mantissas based on the most significant K bits of exponent differences before the full M-bit exponent comparison is complete. This preliminary action uses the available MSB information to start the alignment process early, then refines the shifting based on the remaining LSB comparisons.
Solution Approach 2:
The patent segments the exponent comparison and mantissa shifting operations into two phases: initial shifting based on MSB comparisons, and final adjustment based on LSB comparisons. This segmentation allows overlapping of operations and reduces total latency while maintaining precision.
3Measurement precision
If traditional dot-product computation methods are used, then the calculation is accurate, but the computational intensity is high due to repeated maximum exponent determination and right-shifting operations
Solution Approach 1:
The patent segments the exponent handling into MSB and LSB portions, allowing parallel processing paths where MSB comparisons enable early mantissa shifting while LSB comparisons finalize the alignment. This segmentation transforms a sequential bottleneck into a parallelizable process.
Solution Approach 2:
The patent performs preliminary operations using MSB information (candidate identification, initial shifting) before completing LSB comparisons. This preliminary action enables pipeline overlap and reduces the critical path length, improving throughput while maintaining accuracy.
Data Source
AI summary
Techniques are presented to improve the speed of calculating floating-point dot-products, such as in a floating point unit (FPU). Rather than determine the full maximum exponent initially and wait until the full individual shift amounts are calculated to right-shift each mantissa product, each product of exponents is divided into two fields, a high field and a low field. The low field is used as a fine-grained shift amount to right-shift each mantissa product as soon as the mantissa product is ready, while only hi field participates in the maximum exponent calculation. This allows a dot-product computation to be speed up in two ways: Right-shifting of the mantissa product can begin as soon as the mantissa products are calculated, without waiting for the maximum exponent calculation; and calculation of the maximum exponent is sped up because it is calculated only on the high fields of the exponent, not its full-width.