Floating-Point Addition Rounding Latency Elimination
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current processor architectures face challenges in efficiently handling large matrices and performing floating-point operations, particularly in deep learning contexts, due to the impact of rounding on latency and power consumption.
Innovation Solution
The implementation of a system that utilizes 2-dimensional data structures called 'tiles' for matrix operations, which includes configurations for tile dimensions, data types, and instructions for reduced-latency floating-point addition, allowing parallel rounding and elimination of rounding latency in floating-point arithmetic operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional sequential rounding is used in floating-point addition, then rounding precision is maintained, but execution latency increases
Solution Approach 1:
The patent pre-calculates rounding adjustment values and stores them in lookup tables before the actual floating-point addition is needed. During execution, the pre-computed rounding values are directly retrieved and applied, eliminating the need for sequential rounding calculations and reducing latency while preserving rounding precision.
Solution Approach 2:
The floating-point addition process is segmented into independent parallel paths: one path computes the sum while another path simultaneously computes rounding adjustment values. These segmented paths operate independently and can be executed in parallel, then merged to produce the final rounded result, thereby reducing overall execution latency.
2Measurement precision
If traditional floating-point addition with rounding is used, then accuracy is maintained, but power consumption increases
Solution Approach 1:
Rounding adjustment values are pre-computed and stored in lookup tables during system initialization or idle periods. During actual floating-point operations, the processor directly retrieves these pre-computed values instead of performing energy-intensive rounding calculations in real-time, significantly reducing power consumption while maintaining accuracy.
Solution Approach 2:
The patent replaces the mechanical/computational rounding process with a memory lookup operation. Instead of using complex arithmetic logic to perform rounding, the system substitutes it with a simpler memory access operation that retrieves pre-computed rounding values, thereby reducing the computational energy required while preserving rounding accuracy.
3Measurement precision
If full precision floating-point operations are performed, then computational accuracy is maintained, but throughput decreases
Solution Approach 1:
The processor pipeline is segmented into multiple stages that handle different aspects of floating-point operations simultaneously. Multiple floating-point additions can be initiated in different pipeline stages with different precision requirements, allowing the system to maintain high throughput while preserving accuracy for operations that require it.
Solution Approach 2:
The patent applies rounding adjustments selectively based on the specific operation and its precision requirements. Not all floating-point operations require full precision rounding - the system performs partial rounding or uses pre-computed adjustment values for operations where full precision is not critical, thereby increasing throughput while maintaining sufficient accuracy.
Data Source
Figure 1A~1B
Figure 2(A)~2(C)
Figure 3
AI summary
Disclosed embodiments relate to performing floating-point addition with selected rounding. In one example, a processor includes circuitry to decode and execute an instruction specifying locations of first and second floating-point (FP) sources, and an opcode indicating the processor is to: bring the FP sources into alignment by shifting a mantissa of the smaller source FP operand to the right by a difference between their exponents, generating rounding controls based on any bits that escape; simultaneously generate a sum of the FP sources and of the FP sources plus one, the sums having a fuzzy-Jbit format having an additional Jbit into which a carry-out, if any, select one of the sums based on the rounding controls, and generate a result comprising a mantissa-wide number of most-significant bits of the selected sum, starting with the most significant non-zero Jbit.