Mixed-Precision Fused Multiply-Add Operator for Lossless Accumulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hardware operators for matrix multiplication in artificial intelligence and deep learning face challenges in maintaining precision during the accumulation of partial products, leading to rounding errors and loss of accuracy due to the limited dynamic range and bit size of floating-point registers.
Innovation Solution
A mixed-precision FMA operator is designed with a multiplier and adder configuration that converts floating-point operands into fixed-point numbers with sufficient bit size for accurate accumulation, using alignment circuits to ensure the dynamic range is covered, and a rounding and normalization circuit to maintain precision in the result, even when the exponent exceeds the capacity of the fixed-point representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If floating-point registers with limited bit size are used for accumulation, then device complexity is reduced, but manufacturing precision deteriorates due to rounding errors and loss of accuracy
Solution Approach 1:
The accumulation process is segmented into multiple fixed-point registers (e.g., high part and low part registers) that collectively provide sufficient precision. Each register handles a specific portion of the accumulated value, avoiding the need for a single large floating-point register while maintaining high precision through coordinated management of multiple smaller units.
2Manufacturing precision
If larger fixed-point registers are used to cover full dynamic range, then manufacturing precision is improved, but device complexity increases
Solution Approach 1:
Instead of using one large fixed-point register, the precision requirement is segmented across multiple smaller registers. The high part register stores the most significant bits while the low part register stores the least significant bits, collectively achieving the required precision without the complexity of a single large register.
Solution Approach 2:
The system dynamically selects and manages different register configurations based on the accumulation state. By adaptively handling overflow and underflow between high and low part registers, the system maintains precision flexibility without requiring a fixed large register size for all cases.
3Device complexity
If conventional FMA operators are used, then device complexity is reduced, but measurement precision deteriorates due to rounding errors when adding numbers of opposite sign
Solution Approach 1:
The addition operation is segmented into multiple steps handling different bit ranges separately. By processing high and low parts independently and managing carries between them, the system avoids the rounding errors that occur in conventional single-step additions, particularly when adding numbers of opposite sign.
Solution Approach 2:
The system performs preliminary alignment and preparation of operands before addition, ensuring that high and low parts are properly positioned. This preliminary action prevents loss of precision by establishing the correct bit alignment and magnitude relationships before the actual addition occurs.
Data Source
AI summary
A fused multiply-add hardware operator comprising a multiplier receiving two multiplicands as floating-point numbers encoded in a first precision format; an alignment circuit associated with the multiplier configured to convert the result of the multiplication into a first fixed-point number; and an adder configured to add the first fixed-point number and an addition operand. The addition operand is a floating-point number encoded in a second precision format, and the operator comprises an alignment circuit associated with the addition operand, configured to convert the addition operand into a second fixed-point number of reduced dynamic range relative to the dynamic range of the addition operand, having a number of bits equal to the number of bits of the first fixed-point number, extended on both sides by at least the size of the mantissa of the addition operand; the adder configured to add the first and second fixed-point numbers without loss.