Ultra-low Precision FMMA Unit Parallel Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computing systems face challenges in achieving high throughput and power efficiency for deep learning neural networks due to the computational intensity of floating-point operations, particularly in training systems that require precise calculations across various decimal places, where existing precision formats are either too resource-intensive or inefficient.
Innovation Solution
Implementing an ultra-low precision floating-point fused multiply-multiply-accumulate (FMMA) unit using processors, which aligns mantissas and exponents of operands in parallel to perform operations in reduced precision formats like fp8 and fp16, enabling faster computation without increasing hardware costs or latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional floating-point formats (fp32, fp64) are used to maintain precision in deep learning computations, then measurement precision is improved, but use of energy and computational throughput deteriorate
Solution Approach 1:
The patent changes the precision parameter from conventional fp32/fp64 to ultra-low precision formats (fp8, fp4), fundamentally altering the computational approach. This parameter change enables maintaining acceptable precision for deep learning while dramatically reducing energy consumption and increasing throughput by 4-8x compared to conventional formats
2Use of energy by moving object
If ultra-low precision formats (fp8, fp4) are used to reduce energy consumption, then use of energy is improved, but measurement precision deteriorates
Solution Approach 1:
The patent applies partial precision where different parts of the computational system use different precision levels. Accumulators and intermediate storage use higher precision (fp32) while computation operations use ultra-low precision (fp8, fp4). This partial application of precision maintains overall computational accuracy while achieving energy efficiency benefits
3Measurement precision
If higher precision formats are used to maintain computational accuracy, then measurement precision is improved, but productivity deteriorates
Solution Approach 1:
The patent changes the precision parameter to ultra-low precision formats, which directly increases computational throughput by reducing the computational complexity of arithmetic operations. The fp8 and fp4 formats enable 4-8x higher productivity compared to fp32 while maintaining acceptable precision through the use of accumulators
4Productivity
If reduced precision formats are used to increase throughput, then productivity is improved, but measurement precision deteriorates
Solution Approach 1:
The patent segments the computational process into distinct stages: ultra-low precision computation operations (fp8, fp4) for high throughput, followed by accumulation in higher precision registers (fp32) for precision preservation. This segmentation allows each stage to optimize for its specific function, achieving both high productivity and maintained precision
Data Source
AI summary
Embodiments for implementing a fused multiply-multiply-accumulate (“FMMA”) unit by one or more processors in a computing system. Mantissas for two products, an exponent difference of the two products serving as an alignment shift amount for a product of the two products having a smallest exponent, and an alignment shift amount for an addend relative to an alternative product of the two product having a larger exponent may be determined in parallel. The addend may be aligned relative to the alternative product having the larger exponent. The product having the smallest exponent may be aligned relative to the alternative product having the larger exponent according to the alignment shift amount.