Hybrid Floating-Point Integer Multiply-Accumulate Operator
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional deep learning techniques rely on costly FP32 arithmetic for calculations that modify tensor representations between quantization domains, despite advantages of INT8 arithmetic for resource efficiency.
Innovation Solution
A hybrid hardware multiply-accumulate operator (HFMA) is introduced, capable of multiplying a floating-point number by an integer and adding an integer operand, optimizing operations to reduce the need for FP32 arithmetic.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If FP32 arithmetic is used for calculations modifying tensor representations between quantization domains, then calculation precision is maintained, but hardware resource consumption and power usage increase significantly
Solution Approach 1:
The patent changes the numerical precision parameter from FP32 to INT8 for tensor representations, and uses scaling factors to maintain the required dynamic range. This parameter change allows using simpler integer arithmetic instead of floating-point arithmetic, reducing power consumption while maintaining calculation precision through proper scaling.
Solution Approach 2:
The patent uses simplified INT8 arithmetic operations instead of complex FP32 operations. The INT8 operations are computationally cheaper and can be discarded after each quantization operation, with results converted back to FP32 only when needed, reducing overall hardware resource consumption and power usage.
2Measurement precision
If FP32 arithmetic is used for calculations modifying tensor representations, then accurate results are obtained, but silicon area and performance cost increase
Solution Approach 1:
The patent changes the arithmetic precision parameter from FP32 to INT8, using scaling factors to preserve the dynamic range required for accurate calculations. This allows using faster integer arithmetic units instead of slower floating-point units, improving performance efficiency while maintaining calculation accuracy through proper scaling and rounding.
Solution Approach 2:
The patent segments the calculation process into two parts: INT8 arithmetic operations for efficient computation, and FP32 operations only when needed for final results or intermediate conversions. This segmentation allows the majority of operations to use high-performance integer arithmetic while maintaining accuracy where floating-point precision is required.
3Use of energy by moving object
If INT8 arithmetic is used for resource-efficient operations, then power consumption and silicon area are reduced, but the ability to handle operations between different quantization domains is limited
Solution Approach 1:
The patent introduces scaling factors as intermediary elements that bridge different quantization domains. These scaling factors allow INT8 arithmetic operations to correctly handle tensors with different quantization parameters by applying appropriate scaling before and after operations, maintaining quantization domain flexibility while using power-efficient integer arithmetic.
Solution Approach 2:
The patent dynamically adjusts the scaling factors based on the quantization domains of input tensors, allowing the same INT8 arithmetic unit to handle operations across different quantization domains. This parameter adaptation enables quantization domain flexibility without requiring complex floating-point arithmetic, maintaining power efficiency.
4Ease of manufacture
If conventional quantization with INT8 arithmetic is used, then resource efficiency improves, but operations requiring FP32 arithmetic increase overall computational cost
Solution Approach 1:
The patent changes the arithmetic parameter from FP32 to INT8 for all operations that can be performed with integer arithmetic, including operations between different quantization domains. By using scaling factors to handle domain transformations, the patent eliminates the need for costly FP32 operations, improving resource efficiency and reducing computational time.
Solution Approach 2:
The patent uses disposable INT8 intermediate results that can be computed quickly and discarded after conversion to the required output format. These cheap integer operations replace expensive FP32 operations, reducing both resource consumption and computational time while maintaining the required precision through scaling factors.
Data Source
AI summary
The present disclosure relates to a hardware hybrid multiply-accumulate operator configured to multiply a multiplicand formatted as a floating-point number by a multiplicand formatted as an integer and add an integer-formatted result of the multiplication to an operand formatted as an integer.