Floating-point dot-product hardware with wide multiply-adder tree for machine learning accelerators

Abstract

The invention relates to floating-point dot-product hardware with a wide multiply-adder tree for machine learning accelerators. Systems, apparatuses and methods may provide for technology that conducta first alignment between a plurality of floating-point numbers based on a first subset of exponent bits. The technology may also conduct, at least partially in parallel with the first alignment, a second alignment between the plurality of floating-point numbers based on a second subset of exponent bits, where the first subset of exponent bits are LSBs and the second subset of exponent bits are MSBs. In one example, technology adds the aligned plurality of floating-point numbers to one another. With regard to the second alignment, the technology may also identify individual exponents of a plurality of floating-point numbers, identify a maximum exponent across the individual exponents, and conduct a subtraction of the individual exponents from the maximum exponent, where the subtraction isconducted from MSB to LSB.

Description

technical field [0001] Embodiments relate generally to machine learning. More specifically, embodiments relate to floating point dot product hardware with wide multiply-adder trees for machine learning accelerators. Background technique [0002] Deep neural networks (DNNs) are commonly used in machine learning (ML) workloads to perform matrix multiplication and convolution operations, which tend to be the most power- and performance-limiting operations in ML workloads. Although hardware accelerators with dot-product computation units have been proposed to improve the area and energy efficiency of these operations (eg, using various dataflow architectures and data types), there is still much room for improvement. For example, a traditional floating-point (FP) dot product hardware solution may first find the largest exponent in the floating-point product, and use the largest exponent and the corresponding single exponent to align each product mantissa (e.g., significand, coef...

AI Technical Summary

Problems solved by technology

A global search for the max exponent introduces latency (e.g. thus reducing performance)

Also, alignment can involve a relatively large amount of hardware (such as aligning shifter stages), which adds latency, cost, and / or power consumption

Indeed, as ML applications transition from standard number formats (e.g. floating-point 16-bit / FP16 with 5-bit exponent) to more optimized number formats (e.g. Brain floating-point with 8-bit exponent 16bit / Bfloat16), power and performance limits may increase

Images