BF16 Fused Multiply-Add Instructions for Efficient Neural Network Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning and artificial intelligence applications face inefficiencies due to the computational intensity of deep neural network training, where classical IEEE-754 32-bit (FP32) and 64-bit (FP64) arithmetic is not optimized for reduced precision operations, leading to suboptimal performance in lower-precision arithmetic.
Innovation Solution
Implementation of BFloat16 (BF16) fused multiply-add (FMA) instructions that perform operations on BF16 data elements, allowing for efficient execution of FMA operations with reduced precision while maintaining range and offering regularization properties for deep learning tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If classical IEEE-754 32-bit (FP32) and 64-bit (FP64) arithmetic is used, then precision and reliability are maintained, but computational efficiency and speed are reduced due to the computational intensity of deep neural network training
Solution Approach 1:
The patent changes the precision parameter of floating-point arithmetic from classical FP32/FP64 to BFloat16, reducing the number of bits used for representation. This parameter change enables faster computation in FMA units while maintaining sufficient precision for machine learning applications, directly resolving the contradiction between computational efficiency and arithmetic precision
2Speed
If reduced precision arithmetic is implemented, then computational speed is increased disproportional to shortened width, but range and accuracy may be compromised
Solution Approach 1:
The BFloat16 format applies local quality by preserving the 8-bit exponent field (same as FP32) while reducing the significand to 7 bits. This selective preservation of exponent precision maintains the range and scale characteristics crucial for neural network training, while the reduced significand provides sufficient precision for the computational tasks, achieving both speed and reliability
Solution Approach 2:
The patent modifies the floating-point representation parameters by reducing the significand width from 23 bits (FP32) to 7 bits (BFloat16) while keeping the exponent width constant at 8 bits. This parameter change achieves disproportional speed improvement while maintaining the range characteristics necessary for reliable machine learning computation
Data Source
AI summary
Techniques for performing BF16 FMA in response to an instruction are described. In some examples, an instruction has fields for an opcode, an identification of location of a packed data source/destination operand (a first source), an identification of a location of a second packed data source operand, an identification of a location of a third packed data source operand, and an identification of location of a packed data source/destination operand, wherein the opcode is to indicate operand ordering and that execution circuitry is to, per data element position, perform a BF16 value fused multiply-accumulate operation using the first, second, and third source operands and store a result in a corresponding data element position of the source/destination operand