Floating point dot product operator with correct rounding

By designing a fusion multiplication and addition hardware calculator, rounding in the prior art is solved by using multiplier, alignment circuit and adder to add half-precision product with higher precision without losing accuracy, rounding in the prior art The problems of error and accuracy loss achieve higher calculation accuracy.

CN112130803BActive Publication Date: 2024-11-26KALRAY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010578649.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-06-25
Filing Date
2020-06-23
Publication Date
2024-11-26
Estimated Expiration
2040-06-23

AI Technical Summary

Technical Problem

Existing FMA operators have rounding errors and accuracy losses in mixed precision matrix multiplication, especially when adding half-precision floating point product with higher precision addition operands, resulting in inaccurate results.

Method used

A fusion multiplication and addition hardware operator is designed, including multiplier, alignment circuit and adder, by converting the multiplication result into a fixed number of points with sufficient number of bits and adding operations with higher precision without losing accuracy Add numbers, convert the results into floating-point numbers using rounding and normalization circuits, ensuring that all bits participate in the rounding calculation.

Benefits of technology

It realizes floating point number operation without loss under mixing accuracy, improves calculation accuracy, reduces rounding errors, and ensures the accuracy of the results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112130803B_ABST
    Figure CN112130803B_ABST
Patent Text Reader

Abstract

The present invention relates to a hardware operator for dot product calculation, comprising a plurality of multipliers (10), each multiplier receiving two multiplicands (a, b) in a floating point format encoded in a first precision format (fp16); an alignment circuit (12) associated with each multiplier, configured to convert the result of the multiplication into a corresponding fixed point number with a sufficient number of bits (80) to cover the entire dynamic range of the multiplication based on the exponent of the corresponding multiplicand; and a plurality of adders (30) configured to add the fixed point numbers provided by the multipliers without loss to provide a sum in a fixed point format.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a hardware operator for processing floating point numbers in a processor core, and more particularly to an operator for calculating dot products based on a fused multiply-add operator, generally referred to as FMA. Background Art

[0002] Artificial intelligence techniques, especially deep learning, are particularly demanding in multiplication of large matrices, which can have hundreds of rows and hundreds of columns. Therefore, hardware accelerators that specialize in mixed-precision matrix multiplication are emerging.

[0003] Multiplication of large matrices is usually implemented in blocks, i.e. by decomposing the matrix into sub-matrices of a size suitable for the computing resources. Accelerators are therefore designed to efficiently calculate the product of these sub-matrices. Such accelerators include an operator that can calculate the dot product of the vectors representing the rows and columns of the sub-matrix in one instruction cycle and add the corresponding partial results to the partial results accumulated in the previous cycle. After multiple cycles, the accumulation of the partial results is the dot product of the vectors representing one row and one column of the complete matrix.

[0004] This operator uses FMA technology.

[0005] Figure 1 A conventional FMA operator is schematically shown. The operator typically takes three binary floating point operands, namely two multiplication operands (i.e., multiplicands a and b) and an addition operand c. It computes the term ab+c to produce a result s in a register designated as ACC. The register is designated in this way because it is typically used to accumulate several products over several cycles and reuse the output of the register as the addition operand c in the next cycle, as shown by the dotted line.

[0006] In the article ["Modified Fused Multiply and Add for Exact Low Precision Product Accumulation", Nicolas Brunie, IEEE 24th Symposium on Computer Arithmetic (ARITH), July 2017], the multiplicands a and b are in half-precision floating point format, also known as "binary 16" or "fp16" according to the IEEE-754 standard. A number in "fp16" format has a sign bit, a 5-bit exponent, and a 10+1-bit mantissa (including an implicit bit encoded in the exponent).

[0007] The ACC register is designed to accommodate the full dynamic range of the product ab in fixed-point format. For a multiplicand in "fp16" format, an 80-bit register (plus possibly a few overflow bits) is sufficient, with the fixed point located at position 49 of the register. The format of the addition operand c is the same as the contents of the ACC register.

[0008] As long as the registers do not overflow, this structure makes it possible to obtain an exact result for each operation ab+c and to maintain an exact cumulative result cycle after cycle, thus avoiding round-off errors and precision loss after adding numbers of opposite signs but close absolute values.

[0009] The above article also proposes that in a mixed-precision FMA configuration, at the end of the accumulation phase, the contents of the registers are converted to a higher precision format, such as "binary 32". However, the numbers converted in this way do not cover the full dynamic range of the "binary 32" format because the exponent of the product ab is defined as only 6 bits instead of 8 bits.

[0010] Figure 2 The application of the FMA structure to an accumulation dot product operator is schematically shown, for example as described in U.S. Patent Application 2018 / 0321938. Four pairs of multiplicands (a1, b1), (a2, b2), (a3, b3), and (a4, b4) are provided to corresponding multipliers. Four result products p1 to p4 (called partial products) and an addition operand c are added simultaneously by an adder tree. Both the multiplicand and the addition operand are in the same floating point format. The result of the addition is normalized and rounded to convert to the original floating point format so that it can be reused as operand c.

[0011] To add a partial product and an addition operand, the exponents of the terms are compared so that the mantissas of the terms are aligned with each other. Only a window of significant bits corresponding to the highest exponent is reserved for addition and rounding, with the window corresponding to the size of the adder. As a result, the mantissas of the lower exponent terms are truncated or eliminated all together, resulting in larger errors when the partial products of the two larger exponents cancel each other out.

[0012] The above patent application does not mention any specific size of the adder. In the same context of partial product addition, Nicolas Brunie's 2014 paper entitled "Contributions to computerarithmetic and applications to embedded systems" proposed an adder whose size is equal to twice the size of the partial product mantissa plus two bits of overflow margin, i.e., 98 bits of the multiplicand in binary 32 format. In the same context, patent US8615542, for adding four partial products, provides an adder size equal to only the mantissa size of the multiplicand, i.e., 24 bits of the multiplicand in binary 32 format. Summary of the invention

[0013] A fused multiply-add hardware operator is generally provided, comprising: a multiplier receiving two multiplicands as floating point numbers encoded in a first precision format; an alignment circuit associated with the multiplier configured to convert the multiplication result into a first fixed point number having sufficient bits to cover the entire dynamic range of the multiplication based on an exponent of the multiplicand; and an adder configured to add the first fixed point number and an addition operand. The addition operand is a floating point number encoded in a second precision format having a higher precision than the first precision format, and the operator includes an alignment circuit associated with the addition operand configured to convert the addition operand into a second fixed point number of a reduced dynamic range relative to the dynamic range of the addition operand based on an exponent of the addition operand, the number of bits of the second fixed point number being equal to the number of bits of the first fixed point number extended on both sides by at least the size of the mantissa of the addition operand; and the adder is configured to add the first and second fixed point numbers without loss.

[0014] The operator may also include a rounding and normalization circuit configured to convert the adder result into a floating point number in a second precision format, obtain a mantissa from the most significant bit of the adder result, calculate rounding from the remaining bits of the adder result, and determine an exponent from the position of the most significant bit in the adder result.

[0015] The second fixed point number may be right-extended by a number of bits at least equal to the mantissa size of the addition operand; and the rounding circuitry may compute the rounding using the extended bits of the second fixed point number.

[0016] The operator may be configured to provide the addition operand as a result when an exponent of the addition operand exceeds a capacity of the second fixed point number.

[0017] A related method for fused multiplication and addition of binary numbers includes the steps of: multiplying the mantissas of two floating-point multiplicands encoded in a first precision format; converting the multiplication result into a first fixed-point number having sufficient bits to cover the entire dynamic range of the multiplication result; and adding the first fixed-point number and an addition operand. The addition operand is a floating-point number encoded in a second precision format having a higher precision than the first precision format, and the method further includes the steps of: converting the addition operand into a second fixed-point number having a reduced dynamic range relative to the dynamic range of the addition operand, the number of bits of the second fixed-point number being equal to the number of bits of the first fixed-point number extending on either side by at least the size of the mantissa of the addition operand; and losslessly adding the first and second fixed-point numbers.

[0018] On the other hand, a hardware operator for dot product calculation is generally provided, including multiple multipliers, each multiplier receiving two multiplicands in floating point format encoded in a first precision format; an alignment circuit associated with each multiplier, configured to convert the result of the multiplication into a corresponding fixed point number with sufficient bits to cover the entire dynamic range of the multiplication based on the exponent of the corresponding multiplicand; and multiple adders, configured to losslessly add the fixed point numbers provided by the multipliers to provide a sum in the format of the fixed point number.

[0019] The operator may also include: an input of a floating-point addition operand encoded in a second precision format having higher precision than the first precision format; an alignment circuit associated with the addition operand, configured to convert the addition operand into a fixed-point number of reduced dynamic range relative to the dynamic range of the addition operand based on an exponent of the addition operand, the fixed-point number having a number of bits equal to the number of bits of the fixed-point sum extended on either side by at least the size of the mantissa of the addition operand; and an adder configured to losslessly add the fixed-point sum to the fixed-point number of reduced dynamic range.

[0020] The operator may also include a rounding and normalization circuit configured to convert the adder result into a floating point number encoded in the second precision format, obtain a mantissa from the most significant bit of the adder result, calculate rounding from the remaining bits of the adder result, and determine an exponent from the position of the most significant bit in the adder result.

[0021] The reduced dynamic range fixed point number may be right extended by a number of bits at least equal to the mantissa size of the addition operand; and the rounding circuit may calculate the rounding using the extended bits of the reduced dynamic range fixed point number.

[0022] The operator according to may be configured to provide the addition operand as a result when an exponent of the addition operand exceeds the capacity of the fixed point number with a reduced dynamic range.

[0023] An associated method for computing dot products of binary numbers includes the steps of computing a plurality of multiplications in parallel, each multiplication being from two multiplicands encoded in a first precision format as floating point numbers; converting the results of the corresponding multiplications into corresponding fixed point numbers having sufficient bits to cover the entire dynamic range of the multiplications based on the exponents of the multiplicands of each multiplication; and losslessly adding the fixed point numbers resulting from the multiplications to produce a sum in the fixed point format.

[0024] The method may also include the steps of: receiving a floating-point addition operand encoded in a second precision format having higher precision than the first precision format; converting the addition operand into a fixed-point number having a reduced dynamic range relative to the dynamic range of the addition operand based on an exponent of the addition operand, the fixed-point number having a number of bits equal to the number of bits of the fixed-point sum extended on both sides by at least the size of the mantissa of the addition operand; and losslessly adding the fixed-point sum to the fixed-point number of reduced dynamic range. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Embodiments will be described in the following non-limiting description with reference to the accompanying drawings, in which:

[0026] The previously described Figure 1 A schematic diagram showing a conventional fused multiply-add operator (called FMA);

[0027] The previously described Figure 2 A schematic diagram showing a conventional cumulative dot product operator;

[0028] Figure 3 shows a number in fixed-point format used in a mixed-precision binary16 / binary32 FMA operator;

[0029] Figure 4A A lossless compression technique for dynamic range of fixed-point representation in binary32 format is shown;

[0030] Figure 4B A lossless compression technique for dynamic range of fixed-point representation in binary32 format is shown;

[0031] Figure 5 Indicates the use Figure 4A and Figure 4B A schematic diagram of an embodiment of a mixed-precision FMA operator using a technique to implement correct rounding; and

[0032] Figure 6 Indicates the use Figure 4A and Figure 4B Schematic diagram of a correctly rounded mixed-precision dot product and accumulate operator. DETAILED DESCRIPTION

[0033] In order to improve the computational accuracy during the multiple stages of accumulation of partial products, it is desirable to implement mixed-precision FMA, i.e., the addition operand has a higher precision than the multiplicand. In fact, during repeated accumulation, the addition operand tends to increase continuously, while the partial products remain bounded.

[0034] The above-mentioned IEEE article by Nicolas Bruni proposes a solution that provides accurate computation for multiplicands in binary16 format, whose products can be represented in an 80-bit fixed-point format that is still acceptable for hardware processing within the processing units of the processor core.

[0035] However, the product of two binary16s produces a nonstandard floating point number with a sign bit, 6 exponent bits, and 21+1 mantissa bits, encoded on 28 bits. This format can only be used internally. Then, it is desirable for the addition operand to be in a higher precision standard format. For example, the addition operand can have the next higher precision, namely binary32 with a sign bit, 8 exponent bits, and 23+1 mantissa bits. Thus, the binary32 format requires 277 bits for the fixed point encoding, which is too large for hardware processing in a reduced complexity processor core that is replicated dozens of times in an integrated circuit chip.

[0036] Figure 3 The upper part of shows the fixed-point format that can be used for multiplication of binary 16 multiplicands. The format is represented by an 80-bit register REG80, whose bits are specified by the corresponding exponents of the product. The exponent 0 corresponding to the fixed point is at bit 49. The first bit corresponds to the exponent -48, while the last bit corresponds to the exponent 31.

[0037] The 22-bit mantissa of the product, p(22), is placed in the register so that its most significant bits are located at the positions defined by the exponents of the two multiplicands and by one.

[0038] Figure 3 The lower part of shows the fixed-point format, which can be used for binary 32 operands. This format is implemented by a 277-bit register REG277. The required size is given by the relationship exponent_max - exponent_min + 1 + (mantissa_size - 1).

[0039] The exponent 0 corresponding to the fixed point is at position 150. The first position corresponds to the exponent -149, while the last position corresponds to the exponent 127.

[0040] The 24-bit mantissa of the operand, c(24), is placed in the register so that its most significant bit is at the position defined by the exponent of the operand.

[0041] In order to perform an exact addition of operand c and product p, in principle the maximum adder size should be used, i.e. 277 bits. However, since the result in the standard floating point format is expected, this exact result must be rounded. In this case, the aim is to ensure that the result is correctly rounded, i.e. the rounding calculation takes into account all bits of the exact result.

[0042] To produce a correctly rounded mantissa from an exact result that has more bits than the mantissa, three bits immediately following the mantissa in the exact result are used, called the guard bit G, the round bit R, and the sticky bit S. These three bits determine whether the mantissa is increased according to the rounding mode selected. To ensure the best accuracy after a sequence of signed sums, the "round to nearest" mode is preferred.

[0043] However, it should be noted that the value of the sticky bit S is not strictly the value of the bit following the round bit R—it is the bit that is set to 1 if any bit to the right of the round bit is 1. Therefore, to compute correct rounding in all cases, all bits of the exact result are required.

[0044] However, in order to reduce the size of the adder to a reasonable value, the property is used that when adding an 80-bit fixed-point number to a 277-bit fixed-point number, the large range of 277 bits is not necessary to calculate the correct rounding when converting the result of the addition to a floating-point number. Figure 4A and Figure 4B As shown, two cases can be distinguished.

[0045] Figure 4A The nominal case is shown, in which the operand c and the product p may have mutual influence that affects the result of the addition either directly or through rounding effects. The exponent of operand c is strictly between -74 and 57. (Hereafter, the term "position" is defined relative to the fixed-point format, i.e., the position corresponds to the exponent.)

[0046] For the boundary case of exponent 56, the mantissa c(24) is located in segment [56:33] of the fixed-point format, and there is a guard bit G at position 32 between the least significant bit of the mantissa c(24) and the most significant bit of register REG80.

[0047] When register REG80 contains a positive number, guard bit G is 0. The addition is simply the concatenation of segments [56:32], including the mantissa c(24) and the guard bit, and register REG80. Since the result of the addition should be converted to binary 32, the resulting mantissa is the mantissa c(24), possibly adjusted by rounding. For the "round to nearest" mode, a guard bit G of 0 indicates that no adjustment is required, in which case the mantissa c(24) is used directly in the result of the conversion.

[0048] When the content of register REG80 is negative, the guard bit G receives a sign bit of 1, in which case the mantissa c(24) may need to be adjusted when rounding.

[0049] For the boundary case of exponent -73, the mantissa c(24) is located in segment [73:-96], and there are 24 guard bits [-49:-72] of 0 between the least significant bit of register REG80 and the most significant bit of the mantissa c(24).

[0050] In this case, if operand c is positive, the addition is simply the concatenation of register REG80 and segment [49:-96], including the 24 guard bits set to 0 and the mantissa c(24). Because the result of the addition should be converted to binary 32, the resulting mantissa is usually taken from register REG80. However, when the product is the minimum absolute value of its dynamic range, i.e. 2 -48 The mantissa of the result is taken from the last bit and the next 23 bits of register REG80, which is actually the segment [-48:-71], leaving the guard bit G at position -72 before the mantissa c(24) as 0.

[0051] To round "to nearest", a guard bit G of 0 indicates that no adjustment is to be made, in which case the mantissa taken from register REG80 extended by segment [-49:-71] is used directly for the conversion result.

[0052] If operand c is negative, guard bit G at position -72 receives a sign bit of 1, in which case the fetched mantissa may need to be adjusted when rounding.

[0053] Figure 4B shows the case where operand c has Figure 4A exponent e outside the domain of , that is, e ≥ 57 or e ≤ -74. In this case, the product p and operand c have no mutual effect on the final result provided in binary32 format. For the "round to nearest" mode, either operand c is so large (e ≥ 57) that the product p has no effect and operand c can be provided directly as the final result without addition; or operand c is so small (e ≤ -74) that it has no effect and the contents of register REG80 can be used directly for the final result without addition.

[0054] This will not normally happen unless an appropriate initial value is provided for operand c.

[0055] However, in order to ensure that the rounding is always calculated from the exact value, and thus to ensure that it is format-correct in all cases and all rounding modes, all bits of the mantissa of the operand c and the product p may also be considered in this case. In fact, this case corresponds to the case where both the guard bit G and the rounding bit R are 0, and only the value of the sticky bit S is of interest. For the "round to nearest" and "round to 0" modes, a value of 0 for the single bit G indicates that no adjustment is required and that the value of bit S does not matter. On the other hand, for the "round to infinity" mode, if the sign of the result corresponds to the rounding direction (e.g. a positive result rounded to "positive infinity"), a value of 1 for bit S results in an increase of the mantissa, even if bits G and R are 0.

[0056] Therefore, when e≥57, the sticky bit S takes into account the content of register REG80.

[0057] When e≤-74, the sticky bit S takes into account the mantissa c(24).

[0058] From these elements, we can see that in order to calculate the final result in binary32 floating point format and perform correct rounding, Figure 4A It is sufficient to encode operand c in fixed-point format within the defined reduced range of variation, i.e. on the 153-bit segment [56:-96], and handle out-of-bounds cases in a specific way.

[0059] The size of the adder is effectively only 80 bits on either side extending the size of the mantissa of the result, i.e. 24+80+24=128 bits. The remaining 25 bits on the right of the 153 bits are used only to calculate the rounding that affects the mantissa of the result. The adder stage that processes the 24 least significant bits and the 24 most significant bits of the 128 bits can be simplified by the fact that these bits are fixed for the input receiving the product p. The result of the addition can be represented as a fixed point value on 128+o bits, where o represents a number of bits to take into account possible carryover.

[0060] The mantissa of the final result in binary32 floating-point format is taken from the 24 most significant bits of the addition result, and the exponent of the floating-point result is provided directly by the position of the most significant bit of the mantissa.

[0061] Figure 5 is realized Figure 4A and Figure 4B A block diagram of a mixed-precision FMA operator (fp16 / fp32) of the present invention is shown. It should be noted that the diagram shown here is simplified and only those elements that are helpful for understanding the present invention are shown. Certain elements that are traditionally required to handle the peculiarities of standard floating-point numbers, such as subnormal numbers, undefined numbers (NaN), infinities, etc., are not described.

[0062] The FMA operator includes a floating point multiplication unit FP16MUL, which provides an 80-bit fixed point result. The unit receives two multiplicands a and b in fp16 (or binary 16) format. Each multiplicand has a sign bit S, a 5-bit exponent EXP and a 10+1-bit mantissa MANT (the most significant bit (implicitly at 1) of which is not stored). Both mantissas are provided to the multiplier 10, which calculates the product p as a 22-bit integer. The product p is provided to an alignment circuit 12 controlled by an adder 14, which produces the exponential sum of the multiplicands a and b. The alignment circuit 12 is configured to align the 22 bits of the product p on 80 lines at the positions defined by the sum of the exponents plus 1, as described with respect to Figure 3 The circuit 12 thus converts the floating point result of the multiplication into an 80-bit fixed point number.

[0063] As combined Figure 4A and Figure 4B As discussed, the 80 bits output by the alignment circuit are padded to the left and right with 24 bits of 0 to form a 128-bit fixed point number, which corresponds to the absolute value of the product. The 128-bit absolute value passes through the negation circuit 16, which is configured to reverse the sign of the absolute value when the sign of the multiplicand is opposite. In the case of a negative sign, the negation circuit adds a sign (which is 1) to the left of the 80-bit output of the register. The 128-bit number produced by the negation circuit 16 forms the output of the multiplication unit FP16MUL.

[0064] The addition operand c provided to the FMA operator in fp32 (or binary32) format has a sign bit S, an 8-bit exponent EXP, and a 23+1-bit mantissa MANT. The mantissa is provided to an alignment circuit 18 controlled by the exponent of operand c. Circuit 18 is configured to align the 24 bits of the mantissa on 153 lines at the position defined by the exponent, as combined with Figure 4A As discussed, the circuit 18 thus converts the floating point operand into a 153-bit fixed point number.

[0065] Recall that 153 bits are not sufficient to cover the full dynamics of the exponent of an fp32 number, but only cover exponents between 56 and -73, with the value 56 corresponding to the position of the most significant bit of the 153-bit number. Therefore, circuit 18 can be configured to saturate the exponent at the thresholds 56 and -73. As a result, when the exponent is out of range, the mantissa will stay to the left or right of the 153-bit number. In any case, as combined with Figure 4B As discussed, out-of-range situations are handled differently.

[0066] The number provided by circuit 18 passes through a negation circuit 20 controlled by the sign bit of the operand. Alternatively, circuit 20 may be omitted and at circuit 16 the sign of the product is inverted if the sign of the product is not equal to the sign of operand c.

[0067] The 128-bit adder 22 receives the output of the unit FP16MUL and the 128 most significant bits of the 153-bit signed number provided by the negation circuit 20. The result of the addition is a fixed point number of 128+o bits, where o represents a number of bits to allow for possible carry propagation. The 25 least significant bits of the output of the negation circuit are used downstream for rounding calculations.

[0068] The output of adder 22 is processed by normalization and rounding circuit 24, which has the function of converting the fixed-point result of addition into a floating-point number in fp32 format. Figure 4A and Figure 4B As described above, the mantissa of the fp32 number is taken from the 24 most significant bits of the addition result, and the exponent is determined by the position of the most significant bit of the mantissa in the addition result. In general, the rounding is correctly calculated on the bit immediately following the mantissa in the addition result, followed by the 25 least significant bits of the output of the inversion circuit 20.

[0069] Figure 5 Not shown are possible circuit elements that handle the out-of-range case where the exponent of operand c is greater than or equal to 57 or less than or equal to -74. These elements are not critical in view of the described functionality and there are several possible variations.

[0070] For example, when the exponent is greater than or equal to 57, the circuit 24 takes the left mantissa in the addition result, directly takes the exponent of the operand c (not the position of the mantissa), and calculates the rounding by considering the guard bit G being 0 and using the bits after the mantissa in the addition result to determine the sticky bit S. If the content of the register REG80 is positive, the rounding bit R is considered to be 0, or if the content of the register REG80 is negative, the rounding bit R is considered to be 1.

[0071] When the exponent is less than or equal to -74, circuit 24 may operate as in the nominal case, with the mantissa of operand c remaining to the right in the 25 bits outside the adder, contributing to the value of sticky bit s.

[0072] Of course, if the product is zero, the result of the addition is directly the operand c.

[0073] Figure 6 Indicates the use Figure 4A and Figure 4B Schematic diagram of a correctly rounded mixed-precision dot product and accumulate operator.

[0074] and Figure 5 Compared to the FMA operator, the dot product and accumulate operator is designed to add a number of partial products, such as four here, and an addition operand c. Each partial product is calculated by Figure 5The corresponding unit FP16MUL of the type is calculated. The multiplication result is represented in an 80-bit fixed-point format, where there is no need to fill 24 fixed bits on the left and right. The four fixed-point partial product results are provided to an 80-bit multi-adder 30.

[0075] The multi-adder 30 can have a variety of conventional structures. For four addition operands, a hierarchical structure of three full adders can be used, or a structure based on a carry-save adder (CSA), as described in U.S. Patent Application 2018 / 0321938, except that the addition operands here are 80-bit fixed-point numbers, each of which is large enough to cover the entire dynamic range of the corresponding partial product.

[0076] The result of the multi-adder has exact properties regardless of the value of the partial products. In particular, two large partial products can cancel each other without affecting the accuracy of the result because all bits of the partial products are preserved at that point. In conventional arithmetic units, rounding is performed as soon as the addition of the partial products is completed.

[0077] Furthermore, each multiplication unit FP16MUL is independent of the other multiplication units, since it is not necessary to compare the exponents of the partial products to achieve the relative alignment of the mantissas of the partial products. In fact, each unit is converted into a fixed point format common to all numbers. As a result, it is particularly simple to change the number of multiplication units as required at the design level, since there are no interdependencies between the multiplication units. It is also simple to adapt the structure of the multi-adder to the number of operands, since it is performed according to the system design rules. Therefore, the complexity of the operator can be proportional to the number of multiplication units.

[0078] The result of the addition of partial products may exceed 80 bits. Therefore, the result is encoded on 80+o bits, where o specifies a small number of additional significant bits to accommodate overflow, equal to the base 2 logarithm of the number of partial products to be added plus the sign bit. Thus, for four partial products to be added, o=3.

[0079] The 80+o-bit fixed-point number thus provided by the multi-adder will be added to the addition operand c and converted into a fixed-point number within a limited dynamic range, such as Figure 4A and Figure 4BAs shown. The limited dynamic range is here based on 80+o bits, not 80 bits. Therefore, as shown in the right figure, for the processing of the addition operand c, the alignment circuit 18 performs a conversion to a fixed point number of 153+o bits, and adjusts the downstream processing accordingly. In particular, the 128+o most significant bits are provided to the adder 22 on the addition operand c side. On the partial product sum side, the 80+o bits provided by the multi-multiplier 30 are padded left and right with 24 fixed value bits (0 for a positive result or 1 for a negative result).

[0080] The output of adder 22 is Figure 5 The process is performed except that the number of bits is slightly larger, 128+o2, with o2 comprising o bits and one or more extra bits to accommodate overflow from adder 22.

[0081] Therefore, when converting the result of the final addition to a floating point number, the operator structure only performs a single rounding, and the single rounding can be calculated correctly in all cases.

[0082] Many variations and modifications to the described embodiments will become apparent to those skilled in the art. Figure 5 and Figure 6 The operators shown in are described as combinatorial logic circuits. In a processor core clocked at a relatively high frequency, the latency of an operator purely in combinatorial logic may be too high. In this case, the operators may have a pipeline structure with registers for storing intermediate results, for example, for storing the output numbers of the alignment circuits 12 and 18 (such as in combination with Figure 3 discussed).

[0083] Although mixed-precision operators using fp16 and fp32 formats have been described and these operators are of real interest today in terms of performance / complexity ratio, the principles are theoretically applicable to other standard or non-standard precision formats.

Claims

1. A hardware operator for dot product calculation, comprising: a plurality of multipliers, each multiplier receiving two multiplicands in a floating point format encoded in a first precision format; an alignment circuit associated with each multiplier configured to convert, based on an exponent of the corresponding multiplicand, a result of a multiplication of the corresponding multiplicand into a corresponding fixed point number having a sufficient number of bits to cover a full dynamic range of the multiplication; a multi-adder configured to losslessly add the fixed point numbers provided by the multipliers to provide a sum in a fixed point format; input for floating-point addition operands encoded in a second precision format having greater precision than the first precision format; an alignment circuit associated with the addition operand, configured to convert the addition operand into a fixed point number of reduced dynamic range relative to a dynamic range of the addition operand based on an exponent of the addition operand, the fixed point number having a number of bits equal to a number of bits of the fixed point sum extended on either side by at least the size of the mantissa of the addition operand; and An adder configured to losslessly add a fixed point sum with a fixed point number of reduced dynamic range.

2. The operator according to claim 1 comprises a rounding and normalization circuit configured to convert the adder result into a floating point number encoded in the second precision format by obtaining a mantissa from the most significant bit of the adder result, calculating rounding from the remaining bits of the adder result, and determining an exponent from the position of the most significant bit in the adder result.

3. The operator according to claim 2, wherein: The fixed-point number of reduced dynamic range is extended right by a number of bits at least equal to the size of the mantissa of the addition operand; and The rounding and normalization circuitry uses the extended bits of the fixed point number with reduced dynamic range to calculate the rounding.

4. An operator according to claim 2, configured to provide an addition operand as a result when an exponent of the addition operand exceeds the capacity of the fixed point number with reduced dynamic range.

5. A method for calculating the dot product of binary numbers, comprising the following steps: computing in parallel a plurality of multiplications, each multiplication being from two multiplicands of floating point numbers encoded in the first precision format; Based on the exponent of the multiplicand of each multiplication, converting the result of the corresponding multiplication into a corresponding fixed point number having sufficient bits to cover the entire dynamic range of the multiplication; losslessly adding the fixed point numbers resulting from the multiplications to produce a sum in fixed point format; receiving a floating-point addition operand encoded in a second precision format having greater precision than the first precision format; converting the addition operand into a fixed point number having a reduced dynamic range relative to the dynamic range of the addition operand based on an exponent of the addition operand, the fixed point number having a number of bits equal to the number of bits of the fixed point sum extended on both sides by at least the size of the mantissa of the addition operand; and The fixed point sum is losslessly added to a fixed point number with reduced dynamic range.