Bfloat16 Vector Dot Product Instruction for Single-Step Accumulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies lack an efficient method for performing dot product multiplication of vectors containing 16-bit floating-point elements, requiring multiple instructions to convert and accumulate results, which increases memory utilization and reduces performance and power efficiency.
Innovation Solution
A vector packed data instruction (VDPBF16PS) that performs dot product multiplication of 16-bit floating-point elements in two source vectors, generating a 32-bit destination vector with saturation and accumulation in a single instruction, reducing memory utilization and improving performance and power efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple instructions are used to convert and accumulate dot product results, then computational accuracy is maintained, but memory utilization increases and performance decreases
Solution Approach 1:
The patent combines multiple separate instructions (conversion instruction and accumulation instruction) into a single dot product instruction that performs both operations simultaneously. This merging eliminates the need for separate memory accesses and instruction executions, thereby maintaining computational accuracy while improving performance and reducing memory utilization.
Solution Approach 2:
The instruction performs the conversion of 16-bit floating-point elements to 32-bit elements as a preliminary action within the same execution cycle as the multiplication and accumulation. This preliminary conversion integrated into the single instruction avoids additional memory operations and improves overall computational efficiency.
2Measurement precision
If multiple instructions are used to convert and accumulate dot product results, then computational accuracy is maintained, but memory utilization increases
Solution Approach 1:
By merging the conversion and accumulation operations into a single instruction, the patent eliminates the need for intermediate storage of conversion results in memory. The conversion is performed directly within the execution unit, reducing memory utilization while maintaining the accuracy benefits of using 32-bit elements for accumulation.
3Measurement precision
If multiple instructions are used to convert and accumulate dot product results, then computational accuracy is maintained, but power efficiency decreases
Solution Approach 1:
The patent merges multiple instructions into one, which reduces the total number of execution cycles and memory access operations. This consolidation directly reduces power consumption while preserving the computational accuracy achieved through proper 32-bit accumulation of dot product results.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
Disclosed embodiments relate to systems and methods for performing a floating-point dot product instruction. In one example, a processor includes fetch circuitry to fetch a single instruction having fields to specify an opcode, a writemask, and locations of first source, second source, and destination vectors, decode circuitry to decode the fetched instruction, and execution circuitry to execute the instruction as per the opcode. The writemask is to control whether to mask the destination vector, with masked elements of the destination vector being either zeroed or merged. For elements which are not masked, the opcode is to indicate execution circuitry to generate products of N pairs of 16-bit floating-point elements of the first and second source vectors, and accumulate each product with previous contents of a corresponding single-precision element of the destination vector to produce a corresponding result element. The execution circuitry, in generating products, is to convert each 16-bit floating-point element in each pair to a single precision element by packing the 16 bits of the 16-bit floating-point element into the upper 16 bits of the single precision element, zeroing the lower 16 bits of the single precision element. A format of the 16-bit floating-point elements is bfloat16.