Packed BF16 Scale and Reduction Instructions for Faster AI Compute
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing floating-point representations, such as IEEE-754 32-bit (FP32) and 64-bit (FP64), are inefficient for deep learning tasks due to their computational intensity and limited precision, which affects the performance of machine learning and artificial intelligence applications.
Innovation Solution
Implementing BF16 (Brain Floating-Point 16) instructions for calculating a scale, extracting a reduced argument, and rounding BF16 data elements, which involve single instructions that operate on packed BF16 values, supporting various rounding modes and denormal handling, and translating between instruction set architectures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If IEEE-754 32-bit (FP32) and 64-bit (FP64) arithmetic is used, then precision is maintained, but computational intensity increases and performance decreases
Solution Approach 1:
The patent changes the precision parameter from FP32/FP64 to BF16 format, which has 16 bits total (1 sign bit, 8 exponent bits, 7 fraction bits). This parameter change reduces computational intensity while maintaining sufficient precision for deep learning applications, as evidenced by the improved performance in training neural networks
2Productivity
If reduced precision arithmetic is used, then computational speed increases, but precision is compromised
Solution Approach 1:
The patent adopts BF16 format with specific parameters (1 sign bit, 8 exponent bits, 7 fraction bits) that optimize the balance between speed and precision. The 8-bit exponent provides sufficient range while the 7-bit fraction provides adequate precision for deep learning, achieving faster computation without excessive precision loss
Solution Approach 2:
The patent implements dynamic scaling factors to adjust precision requirements during different phases of deep learning training. By dynamically adjusting the scaling precision based on operational needs, the system maintains computational speed while preserving necessary precision where required
3Productivity
If BF16 format is used, then computational efficiency improves, but range and precision balance must be maintained
Solution Approach 1:
The patent introduces scaling factors as intermediary values that bridge the gap between BF16's limited precision and the requirements of deep learning operations. These scaling factors allow the system to maintain the computational efficiency of BF16 while extending the effective range and precision through mathematical transformation
Solution Approach 2:
The patent combines BF16 format with additional scaling precision components to create a composite numerical representation. This composite approach allows the system to leverage the speed of BF16 while incorporating additional precision layers when needed, achieving a balanced solution for diverse deep learning operations
Data Source
AI summary
Techniques for scale and reduction of BF16 data elements are described. An exemplary instruction includes fields for an opcode, an identification of a location of a first packed data source operand, an identification of a location of a second packed data source operand, and an identification of a packed data destination operand, wherein the opcode indicates that execution circuitry is to perform, for each data element position of the packed data source operands, a floating point scale operation of a BF16 data element of the first packed data source by multiplying the data element by a power of 2 value, wherein a value of the exponent of the power of 2 value is a floor value of a BF16 data element of the second packed data source, and store a result of the floating point scale operation into a corresponding data element position of the packed data destination operand.