BF16 Scale and Reduction Instructions for Deep Learning Range-Speed Tradeoffs

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing floating-point representations like IEEE 754 32-bit (FP32) and 64-bit (FP64) arithmetic are inefficient for deep learning tasks due to their high precision, leading to computational bottlenecks, while IEEE 16-bit (FP16) lacks sufficient range for deep learning training.

Innovation Solution

Implementing BF16 (Brain Floating-Point 16) instructions for calculating scale, extracting reduced arguments, and rounding operations to optimize deep learning tasks, which involve single instructions for packed BF16 data elements, supporting translation across instruction sets and handling denormal numbers.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If IEEE 754 32-bit (FP32) and 64-bit (FP64) arithmetic is used, then precision is maintained, but computational speed decreases and computational bottlenecks occur

Engineering Contradiction:

Improvearithmetic precisionVSAvoidcomputational speed

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The patent changes the precision parameter from standard IEEE 754 FP32/FP64 to BF16 format, which uses 16 bits with 8 bits for exponent and 7 bits for fraction. This parameter change reduces computational complexity and increases speed while maintaining sufficient precision for deep learning applications through specialized scale, reduce, and round instructions

Inventive Principle:

Principle #35Parameter changes

2Productivity

If IEEE 16-bit (FP16) is used, then computational speed increases, but range becomes insufficient for deep learning training

Engineering Contradiction:

Improvecomputational speedVSAvoidnumber range

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The patent modifies the FP16 format parameters to create BF16, increasing the exponent field from 5 bits to 8 bits while reducing the fraction field from 10 bits to 7 bits. This parameter change expands the number range to accommodate deep learning training requirements while maintaining computational speed advantages of 16-bit arithmetic

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent introduces scale, reduce, and round instructions as intermediary operations that work with BF16 data elements. These intermediary instructions handle the transition between different precision levels and manage the extended range capabilities, enabling BF16 to bridge the gap between FP16 speed and FP32/FP64 range requirements

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS20260104894A1Bfloat16 scale and/or reduce instructions

Publication Date: 2026.04.16 INTEL CORP

AI summary

Techniques for scale and reduction of BF16 data elements are described. An exemplary instruction includes fields for an opcode, an identification of a location of a first packed data source operand, an identification of a location of a second packed data source operand, and an identification of a packed data destination operand, wherein the opcode indicates that execution circuitry is to perform, for each data element position of the packed data source operands, a floating point scale operation of a BF16 data element of the first packed data source by multiplying the data element by a power of 2 value, wherein a value of the exponent of the power of 2 value is a floor value of a BF16 data element of the second packed data source, and store a result of the floating point scale operation into a corresponding data element position of the packed data destination operand.