Mantissa Alignment and Rounding for Reduced-Width MAC Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing floating-point arithmetic operations in compute-in-memory devices and ASICs suffer from computational inaccuracy when mantissas are truncated without proper rounding, leading to degraded performance in deep neural networks.
Innovation Solution
A method involving mantissa alignment and rounding is employed, where mantissas are modified to have the same exponent, then truncated with rounding based on the most significant bit of the remainder, preserving computational accuracy with reduced bit-width.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If mantissas are truncated without rounding, then device complexity and power consumption are reduced, but computational accuracy deteriorates
Solution Approach 1:
The patent applies preliminary action by performing rounding on the most significant portion of the mantissa before truncation. The rounding operation is performed on the rounded mantissa using the remainder portion as a rounding reference, which prepares the data in advance to minimize accuracy loss when the final truncation occurs. This preliminary rounding action ensures that the truncated result is as accurate as possible given the reduced bit-width.
2Measurement precision
If mantissas are truncated with rounding, then computational accuracy is maintained, but device complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the mantissa into two distinct portions: the most significant portion (rounded mantissa) and the remainder portion. This segmentation allows the system to process only the necessary most significant bits for computational operations while storing the remainder portion separately as a rounding reference. This segmentation enables accurate rounding without requiring the full original bit-width during computation, thus maintaining accuracy while managing complexity.
3Measurement precision
If full precision mantissas are used, then computational accuracy is maintained, but power consumption and area increase
Solution Approach 1:
The patent applies the taking out principle by extracting only the most significant portion of the mantissa for use in computational operations. The remainder portion is extracted and stored separately as a rounding reference but is not used in the main computation. This extraction allows the system to perform computations with reduced bit-width, significantly lowering power consumption and area requirements while maintaining acceptable computational accuracy through the rounding mechanism.
Data Source
AI summary
In some embodiments, computing a sum of floating-point numbers, such as in multiply-accumulate operations, includes aligning the mantissas of the floating point number by adjusting at least a subset of the mantissas so that the exponents of the floating-point numbers are the same. After the alignment, the most significant portion of each mantissa is rounded depending on the remainder of the mantissa, for example the most significant bit of the remainder. The mantissas are then truncated to the rounded most significant portions. The truncated mantissas are then summed. The mantissas being aligned can be products of mantissas of respective inputs and weights. The sum of the rounded portions in such cases are a result of multiply-accumulate operations, with a reduced bit width.