Floating point operand multiplier that decomposes operands into lower precision floating point numbers
By decomposing FP32 numbers into high and low components and utilizing E9S12 format hardwired circuits and Kulisch type dot product operators, the problem of insufficient hardware resource utilization is solved, achieving efficient and accurate FP32 multiplication operations, which are suitable for artificial intelligence computing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- KALRAY
- Filing Date
- 2025-12-16
- Publication Date
- 2026-06-19
AI Technical Summary
Existing technologies suffer from insufficient utilization of hardware resources when handling large matrix multiplications, especially in artificial intelligence applications. In particular, numerical computation in FP32 format requires additional dedicated arithmetic units, and existing decomposition methods are complex and have high latency.
The FP32 number is decomposed into high and low components using the E9S12 format. It is then decomposed into an internal format with lower precision through hard-wired circuitry. A Kulisch dot product operator is used for multiplication, and a multiplier and exponent calculation circuit are combined to realize an FP32 format multiplier.
It achieves efficient and accurate FP32 multiplication operations, reduces hardware complexity and latency, and supports FP16, BF16 and TF32 formats, making it suitable for artificial intelligence computing.
Smart Images
Figure CN122240060A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a hardware arithmetic unit for processing floating-point numbers in a processor core, and more particularly to a multiplier. Background Technology
[0002] Artificial intelligence technologies, especially deep learning, are particularly demanding when it comes to multiplying large matrices that can have hundreds of rows and columns. This has led to the development of hardware accelerators specifically designed for mixed-precision matrix multiplication. In mixed precision, the multiplied values are typically encoded in 16-bit floating-point format FP16 (Binary 16 of the IEEE-754 standard), while the resulting values are encoded in 32-bit floating-point format FP32 (Binary 32 of the IEEE-754 standard).
[0003] Multiplication of large matrices is typically performed in blocks, that is, by decomposing the matrix into submatrices of a size suitable for computational resources. Therefore, accelerators are designed to efficiently compute the products and sums of these submatrices. Such accelerators include operators capable of computing the dot product of vectors representing the rows and columns of the submatrices in a single pipeline cycle and adding the corresponding partial results to the partial results accumulated during previous cycles. After a number of cycles, the accumulation of partial results forms the dot product of vectors representing the rows and columns of the complete matrix.
[0004] U.S. Patent 11,550,544 describes the principle of such an arithmetic unit capable of accumulating several products of floating-point numbers in FP16 format (i.e., having 1 sign bit, 5 exponent bits, and 10 bits for encoding the mantissa (more precisely, the fractional part of the mantissa)). This arithmetic unit implements the Kulisch technique to perform the accumulation precisely using correct rounding by employing an internal 80-bit fixed-point representation corresponding to the dynamic range of the exponent of the product of the FP16 numbers. In practical applications, such an arithmetic unit may include eight multipliers to perform the dot product of two vectors with eight components in a single pipeline cycle.
[0005] The FP16 format and other non-standard formats, sometimes used (such as TF32 (“Tensor Floating-Point”) or BF16 (“Brain Floating-Point”)), are accurate enough for most artificial intelligence applications that processor cores typically do not integrate underutilized higher-precision arithmetic units. However, in the context of more traditional numerical computation, it is desirable to keep all numbers (operands and results) in a uniform FP32 format. To avoid adding dedicated arithmetic units for this in existing hardware, it has been proposed to decompose each number into two components and use available lower-precision multipliers in a single pipeline cycle to perform the summation of the four partial products of these components. In other words, the multiplication of two higher-precision numbers invokes four lower-precision multipliers to perform the operation in one pipeline cycle.
[0006] For example, patent application EP3800544 proposes using a structure designed to accumulate the product of numbers in a lower-precision floating-point format known as FP21 (1 sign bit, 9 exponent bits, and 11 mantissa bits) to perform the multiplication of normalized FP32 numbers. FP32 numbers have 1 sign bit, 8 exponent bits, and 23 mantissa bits (plus implicit bits encoded in the exponent). Each FP32 number to be multiplied is decomposed into high and low components in FP21 format.
[0007] The FP21 format conforms to the general rules applicable to floating-point numbers, namely that 11 mantissa bits encode the fractional part of the mantissa, and the exponent is encoded as an unsigned integer or "with bias", from which a constant bias must be subtracted to obtain the applicable signed exponent.
[0008] The 11 most significant bits of the exponent and the original mantissa of the FP32 number are copied into the exponent and mantissa of the high component, respectively.
[0009] The low-significant components are obtained as follows. The remaining 12 least significant bits of the original mantissa are converted into a valid FP21 format by normalizing the initial exponent and the offset of the bits taken from the original mantissa. Normalization makes the first most significant bit 1 implicit, so that the 12 bits taken from the original mantissa conform to an 11-bit fractional mantissa with implicit bits.
[0010] Patent application US2023004523 proposes a similar decomposition of the FP32 number into FP21 components.
[0011] These structures introduce complexity and latency due to the operations necessary to convert the 12 least significant bits of the original mantissa into a number conforming to the FP21 format (zero counting, shifting, normalization). Summary of the Invention
[0012] The present invention provides a decomposition circuit for an input binary number in a binary arithmetic unit. The input number has a floating-point format with an exponent and a mantissa, and is implemented as a subnormal number using implicit mantissa bits. The decomposition circuit is hardwired to: decompose the input number into a pair of high and low components, the high and low component pairs having an internal format with lower precision than the format of the input number, wherein the exponent has at least the number of bits of the exponent of the input number plus 1, and the significant numbers have at least half the number of bits of the mantissa plus 1; directly connect the bits of the significant numbers of the high components to the circuit providing the value of the implicit bits and the most significant bit of the mantissa, respectively; directly connect the bits of the significant numbers of the low components to the remaining bits of the mantissa, respectively; assign the exponent of the input number with the increased size of the significant numbers of the components to the exponent of the high components; and assign the unchanged exponent of the input number to the exponent of the low components.
[0013] The decomposition circuit may further include: an adder that receives the exponent of the input number and the magnitude of the significant number of the component, and provides the exponent of the high component; and a comparator that receives the exponent of the input number and the binary value 0, and provides the most significant bit of the significant number of the high component.
[0014] A binary arithmetic unit may include: two inputs for two corresponding incoming floating-point numbers having the same format; an output for a result in the form of a floating-point number encoded in a normalized format; a decomposition circuit of the type described above for each incoming number; and a floating-point arithmetic unit core that provides the result and includes at least four inputs that receive components generated by the decomposition circuit, the arithmetic unit being configured to compute the exponent of the result by compensating for the effect of adding the magnitude of the significant number to the exponent of the higher component.
[0015] The arithmetic unit core may include: four floating-point multipliers, each with two inputs; and multiple adders connected to provide the sum of the multiplier outputs as the result. Decomposition circuitry is hardwired to apply each high and low component to the two distinct inputs of the multiplier, such that each multiplier produces a product of distinct parts of the components.
[0016] Multipliers and multiple adders can be formed as part of a Kulisch-type dot product operator based on the precise internal fixed-point representation of the multiplier's output.
[0017] The dot product operator can be configured to operate in truncated Kulisch mode when the maximum exponent of the product exceeds the capacity of the internal fixed-point representation.
[0018] The input numbers can be in normalized FP32 format, resulting in 9 exponent bits and 12 significant bits for the high and low components; 10 exponent bits and 24 significant bits for the product of the components; an internal fixed-point representation of 80 bits; and multiple adders provide results in FP32 format.
[0019] Also provided is a binary arithmetic unit comprising: at least four inputs to binary operands in an internal floating-point format, the internal floating-point format including 9 exponent bits and 12 bits encoding the complete mantissa; an arithmetic core that performs operations between operands in a hard-wired manner and produces a result encoded in a normalized floating-point format; and exponent calculation circuitry for the result, the exponent calculation circuitry being configured to compensate for the effect of adding the value 12 to the exponents of two operands in the operands.
[0020] The arithmetic unit may include: four floating-point multipliers, each including two inputs to binary operands in an internal floating-point format; a multi-adder connected to provide the sum of the multiplier outputs as the result; and an exponent calculation circuit configured to subtract the value 24 from the exponent calculated for the result. Attached Figure Description
[0021] The embodiments will be illustrated in the following description, provided for illustrative purposes only, with reference to the accompanying drawings, in which: Figure 1 This schematically illustrates a decomposition circuit that decomposes an FP32 number into two components according to the implementation scheme. Figure 2 This is an example based on... Figure 1 A diagram illustrating the decomposition of the multiplicands to calculate the sum of the partial products of the two multiplicands; and Figure 3 This is a block diagram of a multiplier for two FP32 multiplicands that uses two decomposition circuits and a dot product operator. Detailed Implementation
[0022] The following presents a structure for decomposing FP32 numbers into high and low components based on an internal floating-point format obtained via direct hard-wiring, and this structure can be processed with low complexity by existing arithmetic units for FP16 or BF16 formats. As an example, refer to the dot product arithmetic unit of U.S. Patent 11,550,544.
[0023] The internal format corresponds to a format in which the implicit bits of the mantissa of a number conforming to the standard floating-point format are made explicit, thus forming a complete mantissa or significant number. Using such an internal format, the basic concepts of implicit bits or "normal" and "subnormal" numbers no longer exist. The exponent indicates the position of the decimal point while allowing the significant digits before the decimal point to be 0. This does not affect subsequent calculations performed separately on the exponent and significant digits. Therefore, normalization of low-level components becomes unnecessary.
[0024] For FP32 numbers, after specifically including an "unpacking" operation that makes implicit bits explicit, the internal format corresponds to the FP21 format described in the previously mentioned patent application. This internal format is represented as E9S12, which defines a 9-bit exponent and 12-bit significant digits. The sign bit, which must be present, is not indicated by this sign notation.
[0025] The high and low components of the E9S12 format can be applied to any arithmetic unit designed to process FP21 numbers downstream of the unpacking phase. In practice, the unpacking phase of existing arithmetic units can be interchanged with the decomposition phase described herein, and the interchange can be performed in response to the selection of an operating mode (e.g., "FP16 / BF16 mode" or "FP32 mode"). The E9S12 format is relatively versatile because it allows the same arithmetic unit to process FP16, BF16, and TF32 numbers, which have lower precision than FP32 and are commonly used in artificial intelligence.
[0026] Figure 1 This schematically represents the number of FP32s that are expected to be decomposed according to the implementation plan. X The decomposition circuit UPCK.
[0027] Input number X Decomposed into high-floating-point components and low-floating-point components Xh , X l These components use a method including a sign bit. Xhs , X l s 9 index levels Xhe , X l e and 12 significant digits Xhm , X l m The internal format is E9S12. Following general rules applicable to other conceivable formats of input numbers, the size of the exponent in the internal format is equal to the size of the exponent of the input number plus 1, and the number of significant digits is equal to half the number of significant digits of the input number, rounded to the next integer. In other words, the concatenated size of the significant digits of the high and low components is at least equal to the size of the significant digits of the input number.
[0028] Then, the high and low components are generated by the UPCK circuit in the following manner.
[0029] significant number of high components X h m It receives the value of the implicit bit and receives the data via direct hardwire. X decimal end m The following 11 most significant bits. The value of the implicit bits is provided by comparator 10, which receives the number... X index e And 8 bits of binary zero. The comparator provides 1 if they are not equal and 0 if they are equal. In fact, the number is subnormal when the exponent is equal to 0 (with bias).
[0030] significant number of low components X l m Receive data via direct hardwired connection X decimal end m The remaining 12 digits.
[0031] High-weight index X h e Receive a number whose size is increased by the number of significant components. X 8-digit index e .index X h e For example, it is provided by adder 12, which receives the index. e And the binary value 12 (1100). In the case of an initial 8-bit exponent overflow, the additional bits of the exponent of the component are intended to account for the increase of the exponent (+12) of the higher component in the following operation. Although the bias applicable to the 9-bit exponent is 255 (2 9-1 -1), but the bias (127) of the FP32 number is preserved so that it is not necessary to adjust the number. X The exponent of replication. At the end of the operation, a compensation is made by adding 12 to the exponent of the higher component, as will be explained later.
[0032] Low component index X l e Receive numbers right-aligned on the 9 exponent bits. X 8-digit index e The ninth digit is zero.
[0033] sign bit s From numbers X Copy to both components.
[0034] An equivalent result can be obtained by retaining the initial exponent of the high component and subtracting the value 12 from the initial exponent of the low component. However, when the original bias exponent is less than 12, the bias exponent of the low component, which was originally intended to be an unsigned integer, becomes negative. This situation can be handled, but it complicates the circuitry that must be adapted to handle signed bias exponents.
[0035] Figure 2 This demonstrates the application of a decomposition circuit to the multiplication of two FP32 numbers. It illustrates the multiplication of two multiplicands from FP32 format. X and Y The sum of the partial products of the obtained high and low component pairs is calculated. The principle is based on the following relationship, which takes into account the application to the high component... X h and Y hExpansion index (+12): [Mathematical Expression 1] X•Y=(2 12 Xh+X l )•(2 12 Y h +Y l ) =2 24 Xh•Y h +2 12 Xh•Y l +2 12 X l •Y h +X l •Y l Figure 2 The addition of partial products from the last row above is illustrated using relative fixed-point representation. The term "relative representation" implies that the partial products all have the same absolute exponent, which is a number... X and Y The sum of the exponents Xe+Ye As indicated, the absolute exponent corresponds to a partial product. X l •Y l The position of the most significant bit. The last line represents the sum obtained from the partial products, and is therefore the expected product in exact internal format. X•Y .
[0036] Each part product has 24 significant bits, which are shifted right by 0, 12, or 24 bits depending on the effect of the expansion exponent of the higher component involved. The result X•Y It has 48 significant digits, a relative exponent of 24, and an absolute exponent of 24+. Xe + Ye .
[0037] Figure 3 These are two FP32 numbers X and YA schematic diagram of a multiplier that uses a decomposition circuit UPCK for each number and a connected dot product core 30 to perform the summation of partial products. The dot product core includes a set of floating-point multipliers 32 (four of which are shown) to perform the summation of partial products. Component pairs provided by the UPCK circuit ( X h , X l )and( Y h , Y l The inputs are distributed to the four multipliers to produce four partial products.
[0038] Adder 34 is configured to calculate the sum of the products provided by the multipliers. The sum is rounded and normalized to provide results in a normalized floating-point format (e.g., FP32). R To this end, circuit 36 shifts the sum left by the number of consecutive zeros in the most significant bits of the sum. The number of zeros, LZC, is provided by circuit 38, which analyzes the bits of the sum. The shifted sum is provided to rounding circuit 40 to produce a 23-bit mantissa and sign bit in FP32 format. The eight exponent bits of the result are calculated in parallel by circuit 42, which receives the maximum exponent Emax of the product and the number of zeros, LZC.
[0039] These rounding and normalization operations are routine and will not be described in further detail. However, as previously indicated, compensation is performed at this stage for adding the value 12 to the exponent of the higher components. As shown, this compensation can be done by subtracting the value 24 from the exponent calculated by circuit 42, where the value 24 corresponds to the effect on the final exponent of the product of the two higher components.
[0040] The arithmetic unit core 30 can be based on a Kulisch-type architecture described in U.S. Patent 11,550,544. In this case, the product provided by each of the multipliers 32 is encoded in an 80-bit fixed-point format suitable for multiplying numbers in FP16 format with 1 sign bit, 5 exponent bits, and 10 mantissa bits. The size 80 of the fixed-point format is, in principle, suitable for the result provided by the multipliers, which includes 1 sign bit, 6 exponent bits, and 22 significant bits.
[0041] As previously indicated, the product of the components of the FP32 numbers to be multiplied has 1 sign bit, 9 exponent bits, and 24 significant bits, thus requiring a dynamic range and precision that are in principle greater than those of the product of FP16 numbers.
[0042] The Kulisch arithmetic unit of US Patent 11,550,544 can be modified relatively simply with low silicon surface cost to suit the desired operation, while benefiting from FP32 format results with system-accurate and, in many cases, correct rounding. R It would be sufficient to simply add two precision bits to the integer multiplier integrated in the floating-point multiplier 32 and make the alignment circuitry associated with the internal 80-bit representation suitable for handling 9-bit exponents instead of 6-bit exponents.
[0043] It should be noted that a 9th exponent bit has been added to the components to compensate for potential overflow caused by adding 12 to the original 8-bit exponent. The product of the components ideally has a 10-bit exponent to encode the sum of the 9-bit exponents. However, the 10 bits only encode overflow caused by adding the value 24 to the sum of the original 8-bit exponents, resulting in a slightly larger dynamic range than an exponent encoded in 9 bits.
[0044] The multi-adder 34, designed to add 80-bit integers, remains unchanged.
[0045] Introducing a 9-bit exponent instead of a 6-bit exponent in this structure leads to a special case where the largest exponent of the products to be added exceeds +30 (the largest exponent of a product of FP16 numbers) or is less than -28 (the smallest exponent of a product of FP16 numbers). In this case, the arithmetic unit is switched to overflow mode, where the internal 80-bit representation is no longer used in fixed-point mode. Instead, the significant digits of the product with the largest exponent are left-aligned in the 80-bit representation, and the resulting shift is propagated to the other products to be summed. For example, if the largest exponent is +40, 40-30=10 significant digits will overflow to the left of the 80-bit representation and be left-aligned in the 80-bit representation, causing the "fixed" decimal point to shift 10 bits to the right. This shift is then propagated to the other products to be summed to maintain alignment. The final result is calculated directly from the exponent of the product. R The exponent is unaffected by this arbitrary shift of the decimal point. As long as the different significant digits maintain their relative positions, the shift can be arbitrary.
[0046] According to an alternative approach, to avoid dynamically detecting overflow conditions, the arithmetic unit can be configured to programmatically switch between "FP16 computation mode" and "FP32 computation mode," wherein the switching also selects an appropriate unpacking stage. In "FP16 computation mode," the 80-bit fixed-point representation is used as originally intended. In "FP32 computation mode," as described above, the significant numbers of the product with the largest exponent are systematically left-aligned in the 80-bit representation.
[0047] This structure results in truncation of the least significant bits when the largest difference between the exponents of the products to be summed exceeds the capacity of an 80-bit representation. With the mantissa of the product with the largest exponent left-aligned within the 80 bits, the mantissa of the product with the smaller exponent can cross the lower boundary of the 80 bits or be entirely outside the 80 bits. This is known as the operation in the "truncated Kulisch" mode. Truncation occurs more precisely when the largest difference between the exponents of the products exceeds 56 (i.e., 2(FP16 largest exponent - FP16 smallest exponent) - (24 - 22)). Despite truncation, the rounding always has the required precision because the final result is rounded to form an FP32 number, using up to 80 precise bits for the significant numbers. However, certain normalization flags provided by the rounding operation (guaranteeing so-called correct rounding) cannot be computed, such as sticky bits, which consider any bits with a value of 1 located outside the bits used for rounding, especially potential bits that have overflowed to the right of the 80 bits.
[0048] However, these potential drawbacks associated with truncation are avoided in the sum of the partial products of the components of the two FP32 numbers to be multiplied, such as... Figure 3 As indicated in [the document]. In reality, as [the document states]. Figure 2 As shown, partial products have a 12-bit mantissa overlapping in the fixed-point representation, such that the sum of these partial products systematically forms a 48-bit mantissa, the size of which is well-fitted within an 80-bit representation for precise internal operation. Truncation only occurs when the arithmetic unit is used to add two products of FP32 numbers or more. In such cases, truncation can sometimes lead to a lack of internal precision when the numbers have opposite signs and the same exponent. However, in artificial intelligence computation, such cases are rare and result in results so small that they are usually ignored anyway.
[0049] Figure 1 The unpacking circuitry can be used in conjunction with other types of arithmetic cores originally designed to handle numbers with lower precision than FP32, where these arithmetic units are modified at low cost to adapt them to handle inputs in E9S12 format. After the corresponding unpacking, this format of input can indiscriminately receive numbers in FP16 or BF16 format, or high and low components from FP32 numbers. When the number is in FP32 format, the arithmetic unit is further configured to calculate the final exponent by compensating for the effect of adding 12 to the exponent of the high component.
[0050] Therefore, an adder-reduce operator (or multiple adder) of, for example, at least four FP16 numbers can be modified so that it can be modified by placing a variable at the input of the adder according to the formulas used in the original code. Figure 1 The unpacking circuitry for each FP32 number is used to process two FP32 operands. The exponent calculation circuitry of the multi-adder is then further modified to subtract the value 12 from the resulting exponent.
Claims
1. A decomposition circuit (UPCK) for an input binary number in a binary arithmetic unit, the input number having a floating-point format (FP32) with an exponent and a fractional mantissa, and implementing a subnormal number using implicit mantissa bits, the decomposition circuit being hardwired to: Pass in the number ( X Decomposed into high-component and low-component pairs ( X h , X l The high and low component pairs have an internal format with lower precision than the format of the input number, wherein the exponent has at least the number of bits of the exponent of the input number plus 1 (9), and the significant number has at least the number of bits equal to half the number of bits of the decimal mantissa plus 1 (12). The significant number of high-component components ( X h m The bits of the fractional part are directly connected to the circuitry that provides the value of the implicit bits and the most significant bit of the fractional mantissa, respectively. The significant number of the low component ( X l m The bits of the decimal part are directly connected to the remaining bits of the mantissa. The input number (12) increases the size of the effective number of the component. X The index of ) is assigned to the index of the higher component ( X h e );and The input number ( X The unchanged index is assigned to the index of the lower component () X l e ).
2. The decomposition circuit according to claim 1, wherein the decomposition circuit comprises: Adder (12), the adder receives the exponent of the input number ( e The magnitude of the significant number of the components, and the exponent of the high components; and Comparator (10), the comparator receives the exponent of the input number ( e () and binary value 0, and provides the most significant bit of the significant number of the high component.
3. A binary arithmetic unit, the binary arithmetic unit comprising: For two corresponding input floating-point numbers with the same format (FP32) X, Y Two inputs; Output of results in normalized (FP32) format for floating-point numbers; The decomposition circuit (UPCK) according to claim 1 is used for each input number. and A floating-point arithmetic unit core (32) provides the result and includes at least four inputs that receive components generated by the decomposition circuit, the arithmetic unit being configured to calculate the exponent of the result by compensating for the effect of adding the magnitude of the significant number to the exponent of the high component.
4. The arithmetic unit according to claim 3, wherein the floating-point arithmetic unit core comprises: Four floating-point multipliers (32), each of which has two inputs; Multiple adders (34) are connected to provide the sum of the outputs of the multipliers as a result; and The decomposition circuit (UPCK) is hardwired to apply each high and low component to two different inputs of the multiplier, such that each multiplier produces a partial product of the components.
5. The arithmetic unit according to claim 4, wherein the multiplier (32) and the multiple adder (34) form part of a Kulisch-type dot product arithmetic unit based on the exact internal fixed-point representation of the output of the multiplier.
6. The arithmetic unit of claim 5, wherein the dot product arithmetic unit is configured to operate in truncated Kulisch mode when the maximum exponent of the product exceeds the capacity of the internal fixed-point representation.
7. The arithmetic unit according to claim 6, wherein: The input number adopts the standard FP32 format, whereby the high component and the low component have 9 exponent bits and 12 significant bits; The product of the components has 10 exponent bits and 24 significant digits; The internal fixed-point representation is 80 bits; and The multi-adder (34) provides results in FP32 format.
8. A binary arithmetic unit, the binary arithmetic unit comprising: For at least four inputs to a binary operand in an internal floating-point format, the internal floating-point format including 9 exponent bits and 12 bits encoding the complete mantissa; The arithmetic unit core performs operations between the operands in a hard-wired manner and produces results encoded in standard floating-point format (FP32). as well as The exponent calculation circuit for the result is configured to compensate for the effect of adding the value 12 to the exponents of the two operands in the operand.
9. The arithmetic unit according to claim 8, wherein the arithmetic unit comprises: Four floating-point multipliers (32), each of which includes two inputs to binary operands for the internal floating-point format; Multiple adders (34) are connected to provide the sum of the outputs of the multipliers as a result; and The exponent calculation circuit is configured to subtract the value 24 from the exponent calculated for the result.