BF16 Arithmetic Instructions for Deep Learning Precision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current arithmetic systems in machine learning and artificial intelligence, particularly for deep neural networks, face inefficiencies due to the limited precision of IEEE-754 16-bit floating-point numbers, which restricts their range and accuracy in deep learning tasks.
Innovation Solution
The implementation of BF16 (Brain Floating-Point 16) arithmetic, which maintains a similar range to FP32 while reducing precision, allowing for faster computations and improved regularization properties in neural network training by performing arithmetic operations, reciprocal calculations, and supporting denormal inputs without the need for upconversion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If IEEE-754 16-bit floating-point numbers are used, then computation speed is improved, but range and accuracy are restricted
Solution Approach 1:
The patent changes the floating-point format parameters by adopting Brain Float 16 (BF16) with an 8-bit exponent and 7-bit significand, compared to IEEE-754 16-bit format. This parameter change maintains the computation speed advantage while improving the range to match FP32, resolving the contradiction between speed and reliability
2Productivity
If reduced precision arithmetic is used, then computational intensity is improved, but precision is reduced
Solution Approach 1:
The patent modifies the precision parameters by using BF16 format with 7 bits of significand instead of 10 bits in IEEE-754 16-bit format. This provides sufficient precision for deep learning applications while maintaining the computational intensity benefits of reduced precision arithmetic
3Quantity of substance
If FP16 format is used, then memory bandwidth is improved, but range is limited
Solution Approach 1:
The patent changes the exponent field parameters from 5 bits in FP16 to 8 bits in BF16, matching FP32's exponent size. This maintains the memory bandwidth efficiency of 16-bit formats while extending the range to accommodate the wider dynamic range required for deep learning training
Data Source
AI summary
Techniques for performing arithmetic operations on BF16 values are described. An exemplary instruction includes fields for an opcode, an identification of a location of a first packed data source operand, an identification of a location of a second packed data source operand, and an identification of location of a packed data destination operand, wherein the opcode is to indicate an arithmetic operation execution circuitry is to perform, for each data element position of the identified packed data source operands, the arithmetic operation on BF16 data elements in that data element position in BF16 format and store a result of each arithmetic operation into a corresponding data element position of the identified packed data destination operand.