Emulating FP32 Matrix Multiplication via Lower Precision Systolic Arrays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems face performance limitations in floating-point calculations, particularly with Single precision floating General Matrix Multiply (SGEMM) operations, leading to inefficiencies in data processing.
Innovation Solution
Emulating higher precision floating-point operations using lower precision formats, such as FP32, to improve system performance while maintaining calculation accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional floating point calculation operations are used, then calculation accuracy is maintained, but processing performance is insufficient
Solution Approach 1:
The patent emulates FP32 floating point calculations by copying the calculation logic onto lower precision hardware (FP16/BF16 units), performing calculations in lower precision format and then reconstructing the results to match FP32 precision requirements. This allows the system to use faster, more efficient lower precision hardware while maintaining the functional equivalence of higher precision operations.
Solution Approach 2:
The patent changes the precision parameter from FP32 to lower precision formats (FP16/BF16) to improve processing performance. By adjusting the precision level parameter, the system achieves faster execution while maintaining sufficient accuracy for the intended applications, resolving the contradiction between speed and precision.
2Measurement precision
If higher precision floating point formats are used, then calculation accuracy is improved, but processing speed decreases
Solution Approach 1:
The patent copies the FP32 calculation semantics onto FP16/BF16 hardware implementations, preserving the mathematical behavior of higher precision operations while executing them on faster, lower precision hardware. This copying approach maintains computational accuracy for the application domain while achieving superior processing speed.
Solution Approach 2:
The patent applies partial precision (FP16/BF16 instead of full FP32) to achieve sufficient accuracy for many applications while significantly improving speed. This partial action approach provides more than enough precision for typical workloads while enabling faster execution, resolving the speed-accuracy tradeoff.
3Productivity
If lower precision formats are used, then processing performance is improved, but calculation accuracy deteriorates
Solution Approach 1:
The patent copies FP32 calculation results back into FP32 format after performing operations in lower precision, ensuring that the output maintains the required precision level. This copying mechanism allows the system to benefit from faster lower precision hardware while delivering results that match FP32 accuracy requirements.
Solution Approach 2:
The patent incorporates feedback mechanisms where calculation results from lower precision operations are validated and reconstructed to match expected FP32 precision characteristics. This feedback loop ensures that the reduced precision operations maintain sufficient accuracy for the application domain while achieving improved processing efficiency.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Emulating floating point calculation using lower precision format calculations is described. An example of a processor includes a floating point unit (FPU) to provide a native floating point operation in a first precision format; and systolic array hardware including multiple data processing units, wherein the processor is to receive data for performance of a matrix multiplication operation in the first precision format; enable an emulated floating point multiplication operation using one or more values with a second precision format, the second precision format having a lower precision than the first precision format, the emulated floating point multiplication including operation of the systolic array hardware; and generate an emulated result for the matrix multiplication operation