Multiply-accumulate operation method and device for extremely low precision training

Abstract

The invention provides a multiply-accumulate operation method and device for extremely low precision training, and the method comprises the steps: obtaining an operated part and to-be-operated data, and the to-be-operated data comprises one or more of floating point, fixed point and logarithm data formats; if the high-precision floating-point number exists in the data to be calculated, decomposing the high-precision floating-point number into the sum of two low-precision floating-point numbers; floating-point, fixed-point and logarithm data formats in the data to be calculated are decoded to be uniform formats of symbols, indexes and mantissas, and input data in the uniform formats are generated; performing multi-term dot product on the input data with the unified format to realize combined calculation of data formats with different precision; and adding the combined calculation result and the calculated part sum by adopting an advanced random rounding method. The method can support the combined calculation among various formats, reduces the bit width of the accumulator on the premise of maintaining the precision requirement by using an advanced random rounding method, and reduces the operation cost and improves the training efficiency under extremely low precision compared with the existing operation hardware.

Description

Technical Field

[0001] The present invention relates to the field of deep learning hardware acceleration technology, and in particular to a multiplication-accumulation operation method and device for extremely low-precision training. Background Art

[0002] Deep learning has achieved extraordinary performance in various application fields, but the computing resources, time and power consumption in training are too high, so researchers use quantized low-bitwidth data to complete training, which significantly reduces memory usage and saves data movement and processing costs. Recent studies have proven the feasibility of ultra-low precision (less than 8 bits) training, which can improve performance by 4-7 times compared to 16-bit systems, but requires the selection of the optimal data format based on the distribution characteristics of the tensor, introducing the need for multiple precision combination operations.

[0003] Random rounding is also necessary for very low-precision training, which can solve the problem of training stagnation and reduce the accumulator bit width while ensuring accuracy, thereby further saving resources. However, existing computing hardware generally does not support precision below 8 bits, combined operations of multiple precisions, and random rounding, so the benefits of very low-precision training cannot be realized, resulting in low training power consumption and area efficiency. Summary of the invention

[0004] The present invention provides a multiplication-accumulation-addition method and device for extremely low precision training, so as to solve the defect that the benefits of extremely low precision training cannot be realized in the prior art, resulting in low training power consumption and area efficiency, and realize multi-precision combined multiplication-accumulation-addition operation for extremely low precision training.

[0005] The present invention provides a multiplication-accumulation-addition operation method for extremely low-precision training, comprising the following steps.

[0006] Obtaining the calculated partial sum and the data to be calculated, wherein the data to be calculated includes one or more of floating point, fixed point and logarithmic data formats; If there is a high-precision floating-point number in the data to be calculated, decomposing the high-precision floating-point number into the sum of two low-precision floating-point numbers; Decoding the floating point, fixed point and logarithmic data formats in the data to be operated into a unified format of sign, exponent and mantissa, and generating input data in a unified format; Perform multiple dot products on the input data in a unified format to achieve combined calculations of data formats with different precisions; The combined calculation result is added to the calculated partial sum using the advance random rounding method.

[0007] According to the multiplication-accumulation method for extremely low-precision training provided by the present invention, the step of decomposing a high-precision floating-point number into the sum of two low-precision floating-point numbers specifically includes: Decompose the high-precision floating-point number FH into two low-precision floating-point numbers F1 and F2; Among them, in F1, the sign and exponent are the same as those of FH, and the mantissa is the high-order mantissa of FH after rounding; In F2, the sign is the sign of F1 or whether it is rounded, the exponent is the number of leading zeros in the low-order mantissa after the exponent of FH is subtracted from the complement, and the mantissa is the low-order mantissa after the exponent of FH is subtracted from the complement.

[0008] According to the multiplication-accumulation-addition operation method for extremely low-precision training provided by the present invention, the step of decoding the floating-point, fixed-point and logarithmic data formats in the data to be operated into a unified format of sign, exponent and mantissa specifically includes: Decode the floating point formats FP8 and FP6 into 1-bit sign, 5-bit exponent, and 3-bit mantissa, expressed as S1E5M3; Decode the floating point format FP4 into 1-bit sign, 4-bit exponent and 1-bit mantissa, expressed as S1E4M1; Decode the logarithmic format LOG4 into 1-bit sign, 4-bit exponent, and 1-bit mantissa, expressed as S1E4M1; Decode the fixed-point format INT4 into 1-bit sign, 5-bit exponent and 3-bit mantissa, expressed as S1E5M3.

[0009] According to the multiplication-accumulation-addition operation method for extremely low-precision training provided by the present invention, the step of performing a specific number of dot products according to the bit width of each mantissa in the input data specifically includes: According to the bit width of each mantissa in the input data, the high-order multiplication is decomposed into a specific number of low-order multiplications, and after performing a specific number of dot multiplications on each of the low-order multiplications, the exponent in the dot product result is shifted to obtain a dot product result.

[0010] According to the multiplication-accumulation method for extremely low-precision training provided by the present invention, multiple dot products are performed on input data in a unified format, including a mantissa addition process and a normalization process; The mantissa addition process specifically includes: Calculate the sign, mantissa and exponent of the 4 products; Set the products with the same exponent and mantissa but opposite signs to 0; Use a parallel index comparison scheme to find the maximum exponent among the four products; Determine the initial shift amount of each product based on the exponent difference and the multiplication shape; Adjusting the offset of the maximum exponent to make it equal to the offset of the exponent in the dot product result; the offset is obtained according to the number of bits of the exponent and a preset fixed bias value; Based on the offset of the maximum exponent and the initial shift amount of the product corresponding to the maximum exponent, the final shift amount of the product corresponding to the maximum exponent is obtained, and according to the final shift amount and sign of the product corresponding to the maximum exponent, each product is aligned and the complement code is taken; The normalization process specifically includes: Perform addition tree reduction on the dot product result; Normalize the dot product result after the addition tree reduction.

[0011] According to the multiplication-accumulation-addition operation method for extremely low-precision training provided by the present invention, the step of adding the combined calculation result to the calculated partial sum by using the advance random rounding method includes: In the mantissa addition process, a random number is filled into the empty space of the dot product result or the calculated part and the shift result, and the two are added to obtain an early random rounding carry; In the normalization process, the result of the early random rounding is corrected according to the number of leading zeros of the mantissa addition result.

[0012] According to the multiplication-accumulation-addition operation method for extremely low-precision training provided by the present invention, the step of adding the combined calculation result to the calculated partial sum by using the advance random rounding method specifically includes: Adding the dot product result to the calculated part and the addition result to obtain an addition result; An advance random rounding method is adopted to obtain an advance rounding carry; Normalizing the addition result for the second time to generate a sign, an exponent, and a mantissa before rounding; Based on the sign, exponent and mantissa before rounding and the lower 2 bits of the random number, the early rounding carry is corrected to obtain the final rounding carry.

[0013] The present invention also provides a multiplication-accumulation operation device for extremely low-precision training, comprising the following modules: An acquisition module, used for acquiring the calculated partial sum and the data to be calculated, wherein the data to be calculated includes one or more of floating point, fixed point and logarithmic data formats; A decomposition module, used for decomposing the high-precision floating-point number into the sum of two low-precision floating-point numbers if there is a high-precision floating-point number in the data to be calculated; A decoding module, used for decoding the floating point, fixed point and logarithmic data formats in the data to be operated into a unified format of sign, exponent and mantissa, and generating input data in a unified format; The combined calculation module is used to perform multiple dot products on the input data in a unified format to achieve combined calculations of data formats with different precisions; The accumulation module is used for adding the combined calculation result to the calculated partial sum by adopting the advance random rounding method.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the processor implements any of the above-described multiplication-accumulation operation methods for extremely low precision training.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the multiplication-accumulation operation methods for extremely low precision training as described above.

[0016] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-mentioned multiplication-accumulation operation methods for extremely low precision training.

[0017] The multiplication and accumulation method and device for extremely low-precision training provided by the present invention obtains the calculated partial sum and the data to be calculated, and the data to be calculated includes one or more of floating point, fixed point and logarithmic data formats; if there is a high-precision floating point number in the data to be calculated, the high-precision floating point number is decomposed into the sum of two low-precision floating point numbers; the floating point, fixed point and logarithmic data formats in the data to be calculated are decoded into a unified format of sign, exponent and mantissa to generate input data in a unified format; multiple dot products are performed on the input data in the unified format to realize combined calculation of different precision data formats; and the combined calculation result is added to the calculated partial sum by using an advance random rounding method. The present invention can support combined calculations between multiple floating point, fixed point and logarithmic formats, and uses an advance random rounding method to reduce the accumulator bit width while maintaining accuracy, so as to realize support for high precision at a low cost, and improve the training efficiency under extremely low precision compared with the existing computing hardware. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is one of the flow charts of the multiplication-accumulation operation method for extremely low-precision training provided by the present invention.

[0020] Figure 2 This is a diagram of the multi-precision combined multiplication and accumulation operation architecture provided by the present invention.

[0021] Figure 3 This is a schematic diagram of the multi-precision combined operation principle provided by the present invention.

[0022] Figure 4 It is a logic diagram of the decoder provided by the present invention.

[0023] Figure 5 It is a structural diagram of a multi-point product unit provided by the present invention.

[0024] Figure 6 The present invention provides a schematic diagram of the principle of early random rounding for additions with unequal bit widths.

[0025] Figure 7 This is a high-precision decomposition principle diagram provided by the present invention.

[0026] Figure 8 It is a schematic diagram of the structure of the multiplication-accumulation operation device for extremely low-precision training provided by the present invention.

[0027] Fig. 9 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] The present invention is described in detail below in conjunction with the accompanying drawings of the specification. The specific operating method in the method embodiment can also be applied to the device embodiment or the system embodiment. In the description of the present invention, unless otherwise specified, "at least one" includes one or more. "Multiple" refers to two or more. For example, at least one of A, B and C includes: A exists alone, B exists alone, A and B exist at the same time, A and C exist at the same time, B and C exist at the same time, and A, B and C exist at the same time. In the present invention, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a kind of association relationship that describes the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0030] The present invention will be described in detail below in conjunction with specific implementation modes.

[0031] In some specific embodiments of the present invention, Figure 1 As shown, this scheme provides a multiplication-accumulation operation method for extremely low-precision training, including: Step 110: Obtain the calculated partial sum and the data to be calculated, wherein the data to be calculated includes one or more of floating point, fixed point and logarithmic data formats; Step 120: If there is a high-precision floating-point number in the data to be calculated, decompose the high-precision floating-point number into the sum of two low-precision floating-point numbers; Step 130: Decode the floating point, fixed point and logarithmic data formats in the data to be operated into a unified format of sign, exponent and mantissa, and generate input data in a unified format; Step 140, performing multiple dot products on the input data in a unified format to achieve combined calculation of data formats with different precisions; Step 150: Use the advance random rounding method to add the combined calculation result to the calculated partial sum.

[0032] It should be noted that the existing hardware solutions for multiplication and accumulation operations cannot achieve precision below 8 bits or multiple precision combination operations, and cannot perform extremely low precision training through random rounding. Therefore, it is difficult to realize the benefits of extremely low precision training and cannot meet the low cost and high efficiency requirements of extremely low precision training.

[0033] Therefore, the present invention obtains the calculated partial sum and the data to be calculated, decomposes the high-precision floating-point number into the sum of two low-precision floating-point numbers, and decodes the floating-point, fixed-point and logarithmic data formats in the data to be calculated into a unified format of sign, exponent and mantissa, and then performs multiple dot products on the input data in the unified format to realize combined calculation of data formats of different precisions; and adopts the advance random rounding method to add the combined calculation result to the calculated partial sum, thereby reducing the accumulator bit width while maintaining the accuracy requirement, and improving the training efficiency under extremely low precision.

[0034] In some possible implementations of the present invention, the step of decomposing a high-precision floating-point number into the sum of two low-precision floating-point numbers specifically includes: Decompose the high-precision floating-point number FH into two low-precision floating-point numbers F1 and F2; Among them, in F1, the sign and exponent are the same as those of FH, and the mantissa is the high-order mantissa of FH after rounding; In F2, the sign is the sign of F1 or whether it is rounded, the exponent is the number of leading zeros in the low-order mantissa after the exponent of FH is subtracted from the complement, and the mantissa is the low-order mantissa after the exponent of FH is subtracted from the complement.

[0035] Specifically, this embodiment provides an implementation method for decomposing a high-precision floating-point number into the sum of two low-precision floating-point numbers. The high-precision floating-point number is decomposed into the sum of two low-precision floating-point numbers, and the decomposition results are sequentially input into the low-precision calculation unit. After multi-cycle accumulation, a high-precision approximate calculation result is obtained, thereby improving the accuracy range supported by the design with a smaller area cost.

[0036] In a possible embodiment, in step 120, the high-precision floating-point number FH is decomposed into the sum of two low-precision floating-point numbers F1 and F2. Specifically, the sign and exponent of F1 are the same as those of FH, the mantissa is the high-order mantissa of FH after rounding, the sign of F2 is the XOR of the sign of F1 or whether it is rounded, the exponent is the FH exponent minus the leading zero number in the low-order mantissa of FH after complement, and the mantissa is the low-order mantissa of FH after complement.

[0037] In some possible implementations of the present invention, the step of decoding the floating point, fixed point and logarithmic data formats in the data to be operated into a unified format of sign, exponent and mantissa specifically includes: Decode the floating point formats FP8 and FP6 into 1-bit sign, 5-bit exponent, and 3-bit mantissa, expressed as S1E5M3; Decode the floating point format FP4 into 1-bit sign, 4-bit exponent and 1-bit mantissa, expressed as S1E4M1; Decode the logarithmic format LOG4 into 1-bit sign, 4-bit exponent, and 1-bit mantissa, expressed as S1E4M1; Decode the fixed-point format INT4 into 1-bit sign, 5-bit exponent and 3-bit mantissa, expressed as S1E5M3.

[0038] Specifically, this embodiment provides an implementation method for decoding the floating-point, fixed-point and logarithmic data formats in the data to be calculated into a unified format of sign, exponent and mantissa, by decoding the floating-point, fixed-point and logarithmic formats into two unified formats of sign, exponent and mantissa: S1E5M3 (1-bit sign, 5-bit exponent, 3-bit mantissa) or S1E4M1, so that subsequent calculation units can perform calculations in a unified format.

[0039] In detail, in step 130, the floating-point, fixed-point and logarithmic data formats in the data to be calculated are decoded into a unified format of sign, exponent and mantissa to generate input data in a unified format. Specifically, the floating-point formats FP8 and FP6 are decoded into S1E5M3 (1-bit sign, 5-bit exponent, 3-bit mantissa), FP4 is decoded into S1E4M1, the logarithmic format LOG4 can be regarded as E4M0, decoded into S1E4M1, and the fixed-point format INT4 can be regarded as a floating-point number with an exponent of 0, which is decoded into S1E5M3.

[0040] In some possible implementations of the present invention, the step of performing a specific number of dot products according to the bit width of each mantissa in the input data specifically includes: According to the bit width of each mantissa in the input data, the high-order multiplication is decomposed into a specific number of low-order multiplications, and after performing a specific number of dot multiplications on each of the low-order multiplications, the exponent in the dot product result is shifted to obtain a dot product result.

[0041] Specifically, this embodiment provides an implementation method for performing a specific number of dot products according to the bit width of each mantissa in the input data, and determines the number of low-order multiplications obtained by decomposition according to the mantissa bit width, which determines the number of dot products performed.

[0042] In a possible embodiment, 1 / 2 / 4 dot products are performed according to the mantissa bit width to complete the combined calculation of LOG4, FP4 (E3M0, E2M1), FP6 (E3M2, E2M3), FP8 (E5M2, E4M3) and INT4, and the results are accumulated in FP12 or INT8 format.

[0043] The process of performing 1 / 2 / 4 dot products according to the mantissa bit width is as follows: decompose the 4-bit multiplication into 4 2-bit multiplications, and then reduce the product after shifting it left by 4 / 2 / 2 / 0 bits; change the shift amount to 2 / 2 / 0 / 0 bits or 0 / 0 / 0 / 0 bits to implement 2 4×2-bit multiplications or 4 2×2-bit multiplications.

[0044] In step 140, the embodiment of the present invention extends the above idea to mixed operations of multiple precisions such as floating point, fixed point and logarithm, performs multiple dot products on the input data in a unified format, and realizes combined calculations of data formats with different precisions.

[0045] Furthermore, multiple dot products are completed according to the input in a unified format, which specifically includes 5 pipeline stages; the first 3 stages obtain 1 / 2 / 4 dot product results in an intermediate format, which are product decoding and cancellation, multiplication and exponent comparison and order shift, addition tree reduction and product normalization; the last 2 stages obtain the final result of adding partial sums, which are the addition of the dot product intermediate result and the partial sum, normalization of the addition result and early random rounding.

[0046] In some possible implementations of the present invention, multiple dot products are performed on input data in a unified format, including a mantissa addition process and a normalization process; The mantissa addition process specifically includes: Calculate the sign, mantissa and exponent of the 4 products; Set the products with the same exponent and mantissa but opposite signs to 0; Use a parallel index comparison scheme to find the maximum exponent among the four products; Determine the initial shift amount of each product based on the exponent difference and the multiplication shape; Adjusting the offset of the maximum exponent to make it equal to the offset of the exponent in the dot product result; the offset is obtained according to the number of bits of the exponent and a preset fixed bias value; Based on the offset of the maximum exponent and the initial shift amount of the product corresponding to the maximum exponent, the final shift amount of the product corresponding to the maximum exponent is obtained, and according to the final shift amount and sign of the product corresponding to the maximum exponent, each product is aligned and the complement code is taken; The normalization process specifically includes: Perform addition tree reduction on the dot product result; Normalize the dot product result after the addition tree reduction.

[0047] Specifically, this embodiment provides an implementation method for performing multiple dot products on input data in a unified format, dividing the operation of multiple dot products into two processes: mantissa addition and normalization, and finally obtaining the dot product result through the two processes.

[0048] In a possible embodiment, the signs of four products are obtained by XORing the input signs, and the exponents of the input multipliers are added to obtain four product exponents.

[0049] It should be noted that the mantissa addition process and the normalization process in this embodiment can be regarded as the operation performed by the first three pipeline stages. Specifically, the first three pipeline stages obtain the dot product result in the intermediate format, which specifically includes: firstly, calculating the signs, mantissas and exponents of the four products, and obtaining the signs of the four products by XOR of the input signs, and adding the exponents of the input multipliers to obtain the four product exponents. Then, the products with the same exponent and mantissa but opposite signs are set to 0, and then the maximum exponent of the four products is compared in parallel, and the shift amount of each product is determined and complemented according to the exponent difference and the multiplication shape, and finally the dot product result after the addition tree reduction is normalized.

[0050] Specifically, if the current mode is to calculate the sum of 4 FP4×FP4, each product represents one FP4×FP4; if the current mode is to calculate one FP8×FP8, each product represents 1 / 4 of FP8×FP8, and each product is shifted and then reduced to generate an FP8×FP8 result.

[0051] In some possible implementations of the present invention, the step of adding the combined calculation result to the calculated partial sum using an advance random rounding method includes: In the mantissa addition process, a random number is filled into the empty space of the dot product result or the calculated part and the shift result, and the two are added to obtain an early random rounding carry; In the normalization process, the result of the early random rounding is corrected according to the number of leading zeros of the mantissa addition result.

[0052] Specifically, this embodiment provides an implementation method for adding the combined calculation result to the calculated partial sum. By using an advance random rounding method for addition of unequal bit widths in the accumulation stage, training stagnation is avoided, and the accumulator bit width can be reduced while ensuring accuracy to further save resources.

[0053] It should be noted that the process of adding the combined calculation result to the calculated partial sum in this embodiment can be regarded as the operation performed by the last two pipeline stages, and the last two pipeline stages obtain the final result of adding the partial sum, specifically including: first, the partial sum is decoded, the exponent is compared and aligned, and then added to the dot product result, and random rounding is started in advance, and then the addition result is normalized, and the previous random rounding result is corrected to obtain the final rounding carry.

[0054] In some possible implementations of the present invention, the step of adding the combined calculation result to the calculated partial sum by using the advance random rounding method specifically includes: Adding the dot product result to the calculated part and the addition result to obtain an addition result; An advance random rounding method is adopted to obtain an advance rounding carry; Normalizing the addition result for the second time to generate a sign, an exponent, and a mantissa before rounding; Based on the sign, exponent and mantissa before rounding and the lower 2 bits of the random number, the early rounding carry is corrected to obtain the final rounding carry.

[0055] Specifically, this embodiment provides a specific implementation method for adding the combined calculation result to the calculated partial sum, obtaining an early rounding carry by adopting an early random rounding method, correcting the early rounding carry by the sign, exponent and mantissa before rounding, and obtaining the final rounding carry.

[0056] Specifically, in step 150, the process of adding the combined calculation result to the calculated partial sum adopts an advance random rounding method for addition of unequal bit widths, specifically including, in the process of mantissa addition, filling the random number into the empty space of the dot product or the partial sum shift result, adding the two to obtain the advance random rounding carry, and correcting the advance random rounding result according to the number of leading 0s of the mantissa addition result in the normalization stage.

[0057] The processes of the above embodiments are described in detail below through specific embodiments.

[0058] In a possible embodiment, Figure 2 This is a diagram of the multi-precision combined multiplication and accumulation operation architecture provided by the present invention, such as Figure 2 As shown in FIG. 1 , the multi-precision combined multiplication-accumulation operation architecture consists of four parts, including a high-precision analog controller, a left / right decoder, a multi-number dot product unit, and a random number generator.

[0059] Specifically, the high-precision analog controller: decomposes the input FP12 into the sum of 2 FP8s, and simulates high-precision calculations by the accumulation of multiple low-precision calculations; the left / right decoder: decodes the floating-point, fixed-point, and logarithmic formats into a unified format of sign, exponent, and mantissa; the multi-point product unit: completes multiple dot products in a unified format, and the output format is INT8 when the input is all INT4, otherwise the output format is FP12; the random number generator: uses a linear feedback shift register (LSFR) to generate a 12-bit random number for random rounding.

[0060] That is, in step 120, if there is a high-precision floating-point number in the data to be calculated, the process of decomposing the high-precision floating-point number into the sum of two low-precision floating-point numbers can be performed by Figure 2 In step 130, the process of decoding the floating point, fixed point and logarithmic data formats in the data to be operated into a unified format of sign, exponent and mantissa, and generating input data in a unified format can be achieved by Figure 2 In step 140, multiple dot products are performed on the input data in a unified format to realize the combined calculation process of data formats with different precisions, which can be achieved by Figure 2 In step 150, the process of adding the combined calculation result to the calculated partial sum by using the advance random rounding method can be realized by Figure 2 The random number generator in is used to assist in the implementation.

[0061] In this architecture, according to the mantissa bit width of the input data, after each operation completes 1 / 2 / 4 dot products, the floating-point format is accumulated under FP12, and the fixed-point format is accumulated under INT8, as shown in the following formula: (1) Among them, F is the result of the operation, is the calculated partial sum, , is the data to be calculated, and K is the number of dot products executed in each calculation.

[0062] In a possible embodiment, as shown in Table 1, Table 1 is a table of precision combination types supported by an embodiment of the present invention, and the table shows the precision modes supported by the architecture.

[0063] Table 1

[0064] Taking K = 1 / 3 and the precision combination FP12×FP12 as an example, one FP12 is decomposed into the sum of two FP8s. Therefore, one FP12×FP12 can be decomposed into four FP8×FP8s accumulated on FP12, that is: FP12 A×FP12 B =FP8 AH ×FP8 BH +FP8 AH ×FP8 BL +FP8 AL ×FP8 BH +FP8 AL ×FP8 BL ; Among them, FP8 AL ×FP8 BL It has almost no effect on the result and can be omitted. Therefore, three FP8 dot products are required to obtain one FP12×FP12, K=1 / 3. The calculation of FP12×FP8 is similar, but only two multiplications and accumulations are required. The high-precision analog controller implements FP12 decomposition and sequentially inputs the decomposition results into the low-precision multi-dot product unit to complete the accumulation to achieve high-precision simulation.

[0065] In a possible embodiment, Figure 3 This is a schematic diagram of the multi-precision combined operation principle provided by the present invention, such as Figure 3 As shown, any 2 n All fixed-point multiplications of 2 bits are decomposed into the sum of multiple 2-bit multiplications after shifting. For example, to implement 4-bit multiplication, the input is split into high 2 bits and low 2 bits, and then 4 2-bit multiplications are performed. The product is then shifted left by 4 / 2 / 2 / 0 bits, and finally an 8-bit addition tree is used for reduction. Similarly, the above circuit can also implement 2 4×2-bit multiplications or 4 2×2-bit multiplications by simply replacing the shift amounts with 2 / 2 / 0 / 0 and 0 / 0 / 0 / 0.

[0066] The solution of the above embodiment can be extended to floating point operations. Since FP8 contains a maximum of 4 mantissas and FP4 / LOG4 contains a maximum of 2 mantissas, Figure 3 Variants of the circuit in the figure can implement 1x FP8×FP8 or 2x FP8×FP4 or 4x FP8×FP4. The required changes include: complementing the product according to the sign, shifting according to the exponent difference, adding floating-point-specific decoding, normalization and rounding, etc.

[0067] In a possible embodiment, Figure 4 is the decoder logic diagram provided by the present invention, such as Figure 4 As shown, the decoder decodes multiple basic types into sign, exponent and mantissa forms, where FP4 is decoded as E4M1 to be compatible with LOG4 (E4M0) and E2M1. FP8 is decoded as E5M3 to be compatible with E5M2 and E4M3. Figure 4 Taking floating point decoding as an example, a fixed point is regarded as a floating point number with an exponent of 0.

[0068] Specifically, the 16-bit input of the decoder consists of 4 FP4 or 2 FP8. In some cases, it is not necessary to use all the data, such as Figure 4 (a) is shown. The result of symbol decoding is the symbol of 4 products, as shown in Figure 4 (b) When decoding the exponent, the exponent bias needs to be adjusted. For example, when decoding E3M0 (bias = 3) to E4M1 (bias = 7), the exponent needs to be increased by 4. Some FP8 exponent decoding results are invalid and are set to 0, such as Figure 4 (c) The 8-bit result of the mantissa decoding contains the leading bit. To implement the 4-bit multiplication required by FP8, we follow Figure 3 After the logic of the 4-bit mantissa is decomposed into four 2-bit mantissas, different arrangements are used in the left / right decoder, such as Figure 4 (d) shown.

[0069] In a possible embodiment, Figure 5 is a multi-point product unit structure diagram provided by the present invention, such as Figure 5 As shown in the figure, the multi-point product unit includes 5 pipeline stages. The first 3 stages obtain the 1 / 2 / 4 dot product results in the intermediate format, and the last 2 stages add them to the partial sum to obtain the final result. Fixed-point and floating-point operations reuse all calculation logic and are not shown in the figure.

[0070] The first stage of the pipeline completes product decoding and cancellation, specifically: first calculate the signs, mantissas and exponents of the four products, obtain the signs of the four products by XORing the input signs, and add the exponents of the input multipliers to obtain the four product exponents. Then, set the products with the same exponent and mantissa but opposite signs to 0, thus avoiding the loss of precision caused by two opposite large products shifting the small product to the right.

[0071] The second stage pipeline completes the multiplication, exponent comparison, and alignment shift. Specifically, the parallel exponent comparison scheme is used to find the maximum exponent among the four products, and the shift amount is determined based on the exponent difference and the multiplication shape. The offset of the maximum exponent is then adjusted to be equal to the offset of the result exponent. Finally, the complement is taken after the alignment product based on the shift amount and sign.

[0072] The third stage pipeline completes the reduction and normalization of the dot product results. Specifically, the dot product results after the addition tree reduction need to be normalized for subsequent standard floating-point addition. The upper 9 bits of the normalized mantissa result are used for floating-point addition, and the lower 5 bits are used for random rounding (SR) performed in advance.

[0073] The fourth stage pipeline completes the addition of the dot product result and the partial sum, specifically: the partial sum is added to the dot product result after decoding, exponent comparison and alignment. The advance SR is performed in parallel with the addition, generating an advance rounding carry. The upper 10 bits of the addition result are used for normalized left shift, and the lower 6 bits are used for SR correction.

[0074] The fifth stage of the pipeline completes the normalization and rounding of the addition result. Specifically, the addition result is normalized for the second time to generate the sign, exponent, and mantissa before rounding. SR correction is performed through the normalization result, the lower 2 bits of the random number, and the early rounding carry to obtain the final rounding carry. The output of non-standard numbers is supported in the rounding stage.

[0075] It is worth noting that traditional stochastic rounding (SR) is performed after normalization, resulting in a large delay. In order to optimize the delay, (Sami Ben Ali et al. 2024. A Stochastic Rounding-Enabled Low-Precision FloatingPoint MAC for DNN Training. In 2024 Design,Automation Test in Europe Conference Exhibition (DATE). 1–6.) proposed an eager-SR strategy of performing SR before normalization and performing SR correction during normalization, but this method is only applicable to floating-point additions with equal addend bit widths.

[0076] Considering that the bit widths of the dot product result (PROD) and the partial sum (PSUM) in the multiplication and accumulation are usually not equal, the present invention proposes an early SR strategy (collectively referred to as Variable-SR) for additions with unequal bit widths, such as Figure 5 As shown in the figure, performance improvement is achieved by optimizing latency and reducing normalized bit width through early SR.

[0077] In a possible embodiment, Figure 6 This is a schematic diagram of the principle of early random rounding of addition with unequal bit widths provided by the present invention, such as Figure 6 As shown, Figure 6 (a) shows how the Variable-SR strategy performs early SR when adding the mantissa. Specifically, after PROD or PSUM is aligned, a 10-bit random number is filled into the shift result. The upper 8 bits of the shift result are used for mantissa addition, and the lower 10 bits are used for early SR addition. The two are completed in parallel. When PROD is aligned, the random number is filled to the right of the PSUM LSB and SR addition is performed with the lower bit of PROD. When PSUM is aligned, the lower 5 bits of the random number are filled into the right of the LSB of PROD, the first SR addition is performed, and then the addition result and the upper 5 bits of the random number are added for the second SR to obtain the early SR rounding carry (S1S2).

[0078] Further, Figure 6(b) shows how the Variable-SR strategy performs SR correction during normalization. Specifically, SR correction is performed in the following four cases according to the number of leading 0s cnt in the result of the mantissa addition: 1) cnt=0: The mantissa addition generates a carry, and the lower 2 bits of the addition result, the 2-bit random number, are added to S1S2 to obtain the final rounded carry (C1C2).

[0079] 2) cnt=1: The addition of the mantissa does not generate a carry. The LSB of the addition result, the 1-bit random number, is added to S1S2 to obtain C1C2.

[0080] 3) cnt=2: The addition of the mantissa produces a 1-bit carry, and C1C2 is equal to S1S2.

[0081] 4) cnt≥3: The addition of the mantissa produces cancellation, the addition result is accurate, and SR is not required. According to the difference in exponents between PROD and PSUM, the lower 5 / 6 bits of PROD are placed in the lower bits of the normalization shifter to ensure accuracy without loss.

[0082] In a possible embodiment, Figure 7 This is a high-precision decomposition principle diagram provided by the present invention, such as Figure 7 As shown, one FP12 is decomposed into the sum of two FP8s, that is, FP12=FP8 H +FP8 L , so one FP12×FP12 can be decomposed into 4 FP8×FP8 accumulations on FP12, that is: FP12 A ×FP12 B =FP8 AH ×FP8 BH +FP8 AH ×FP8 BL +FP8 AL ×FP8 BH +FP8 AL ×FP8 BL , Among them, FP8 AL ×FP8 BL It has almost no effect on the result, so it is omitted. The calculation of FP12×FP8 is similar, but only two multiplications and accumulations are required. The high-precision analog controller implements FP12 decomposition and sequentially inputs the decomposition results into the low-precision multi-point product unit to complete the accumulation to achieve high-precision simulation.

[0083] The specific decomposition plan is: Shilling FP8 H The sign, exponent, and first two bits of the mantissa are the same as those of FP12, and then the third bit R of the mantissa of FP12 is used to calculate the value of FP8. HRound to the nearest integer. By adjusting FP8 L The exponent of FP12 can represent the low-order mantissa of FP12. When R=0, the sign is the same as FP12, the leading 0 count (LZC) is performed on the last 4 bits of FP12, and the leading 1 is used as FP8 L The implicit bit of the FP12 is used as the mantissa. When R=1, the sign is opposite to that of FP12, and the last 4 bits of FP12 are complemented and then LZC is performed.

[0084] The above decomposition is completely accurate when the FP12 exponent is greater than or equal to 7, otherwise it will be due to the theoretical FP8 L The exponent may be less than 1 and cannot be used. We support FP8 L The denormalized number representation reduces the minimum FP12 exponent that can be accurately decomposed from 7 to 5.

[0085] The multiplication-accumulation operation method for extremely low-precision training provided in an embodiment of the present invention supports mutual calculations between multiple floating-point, fixed-point and logarithmic formats, decomposes a high-precision floating-point number into the sum of two low-precision floating-point numbers, and simulates high-precision calculations by the accumulation of multiple low-precision calculations; and utilizes an advance random rounding method to reduce the accumulator bit width while maintaining the accuracy requirement, and also designs a high-precision analog controller to support high precision at a low cost, thereby improving the training efficiency under extremely low precision compared to existing computing hardware.

[0086] The multiplication-accumulation-addition operation device for extremely low precision training provided by the present invention is described below. The multiplication-accumulation-addition operation device for extremely low precision training described below and the multiplication-accumulation-addition operation method for extremely low precision training described above can be referenced to each other.

[0087] In some specific embodiments of the present invention, Figure 8 As shown, this solution provides a multiplication-accumulation operation device for extremely low-precision training, including: An acquisition module 81 is used to acquire the calculated partial sum and the data to be calculated, wherein the data to be calculated includes one or more of floating point, fixed point and logarithmic data formats; A decomposition module 82 is used for decomposing the high-precision floating point number into the sum of two low-precision floating point numbers if there is a high-precision floating point number in the data to be calculated; A decoding module 83, used for decoding the floating point, fixed point and logarithmic data formats in the data to be operated into a unified format of sign, exponent and mantissa, and generating input data in a unified format; A combined calculation module 84 is used to perform multiple dot products on input data in a unified format to achieve combined calculations of data formats with different precisions; The accumulation module 85 is used to add the combined calculation result and the calculated partial sum by adopting an advance random rounding method.

[0088] The implementation principle and beneficial effects of the multiplication-accumulation-addition operation device for extremely low precision training provided in the embodiment of the present invention are similar to the implementation principle and beneficial effects of the multiplication-accumulation-addition operation method for extremely low precision training shown in the above embodiment. Please refer to the implementation principle and beneficial effects of the multiplication-accumulation-addition operation method for extremely low precision training shown in the above embodiment, and no further details will be given here.

[0089] Fig. 9 An example of a physical structure diagram of an electronic device is shown in FIG. Fig. 9 As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930 and a communication bus 940, wherein the processor 910, the communication interface 920 and the memory 930 communicate with each other through the communication bus 940. The processor 910 may call the logic instructions in the memory 930 to execute the multiplication and accumulation method for extremely low-precision training, the method comprising: obtaining the calculated partial sum and the data to be calculated, the data to be calculated comprising one or more of floating point, fixed point and logarithmic data formats; if there is a high-precision floating point number in the data to be calculated, decomposing the high-precision floating point number into the sum of two low-precision floating point numbers; decoding the floating point, fixed point and logarithmic data formats in the data to be calculated into a unified format of sign, exponent and mantissa, generating input data in a unified format; performing multiple dot products on the input data in the unified format to realize the combined calculation of different precision data formats; using the advance random rounding method to add the combined calculation result to the calculated partial sum.

[0090] In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0091] On the other hand, the present invention also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the multiplication and accumulation operation methods for extremely low precision training provided by the above methods, and the method includes: obtaining the calculated partial sum and the data to be calculated, and the data to be calculated includes one or more of floating point, fixed point and logarithmic data formats; if there are high-precision floating-point numbers in the data to be calculated, decomposing the high-precision floating-point numbers into the sum of two low-precision floating-point numbers; decoding the floating point, fixed point and logarithmic data formats in the data to be calculated into a unified format of sign, exponent and mantissa, and generating input data in a unified format; performing multiple dot products on the input data in the unified format to realize combined calculation of data formats of different precisions; using an advance random rounding method to add the combined calculation result to the calculated partial sum.

[0092] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the multiplication and accumulation operation methods for extremely low precision training provided by the above-mentioned methods, the method comprising: obtaining a calculated partial sum and data to be calculated, the data to be calculated comprising one or more of floating point, fixed point and logarithmic data formats; if there is a high-precision floating point number in the data to be calculated, decomposing the high-precision floating point number into the sum of two low-precision floating point numbers; decoding the floating point, fixed point and logarithmic data formats in the data to be calculated into a unified format of sign, exponent and mantissa, and generating input data in a unified format; performing multiple dot products on the input data in the unified format to realize combined calculation of data formats of different precisions; and adding the combined calculation result to the calculated partial sum using an advance random rounding method.

[0093] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0094] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multiplication-accumulation method for extremely low-precision training, characterized in that: include: Obtaining the calculated partial sum and the data to be calculated, wherein the data to be calculated includes one or more of floating point, fixed point and logarithmic data formats; If there is a high-precision floating-point number in the data to be calculated, decomposing the high-precision floating-point number into the sum of two low-precision floating-point numbers; Decoding the floating point, fixed point and logarithmic data formats in the data to be operated into a unified format of sign, exponent and mantissa, and generating input data in a unified format; Perform multiple dot products on the input data in a unified format to achieve combined calculations of data formats with different precisions; The combined calculation result is added to the calculated partial sum using the advance random rounding method.

2. The multiplication-accumulation method for extremely low-precision training according to claim 1, characterized in that: The steps of decomposing a high-precision floating-point number into the sum of two low-precision floating-point numbers include: Decompose the high-precision floating-point number FH into two low-precision floating-point numbers F1 and F2; Among them, in F1, the sign and exponent are the same as those of FH, and the mantissa is the high-order mantissa of FH after rounding; In F2, the sign is the sign of F1 or whether it is rounded, the exponent is the number of leading zeros in the low-order mantissa after the exponent of FH is subtracted from the complement, and the mantissa is the low-order mantissa after the exponent of FH is subtracted from the complement.

3. The multiplication-accumulation method for extremely low-precision training according to claim 1, characterized in that: The step of decoding the floating point, fixed point and logarithmic data formats in the data to be operated into a unified format of sign, exponent and mantissa specifically includes: Decode the floating point formats FP8 and FP6 into 1-bit sign, 5-bit exponent, and 3-bit mantissa, expressed as S1E5M3; Decode the floating point format FP4 into 1-bit sign, 4-bit exponent and 1-bit mantissa, expressed as S1E4M1; Decode the logarithmic format LOG4 into 1-bit sign, 4-bit exponent, and 1-bit mantissa, expressed as S1E4M1; Decode the fixed-point format INT4 into 1-bit sign, 5-bit exponent and 3-bit mantissa, expressed as S1E5M3.

4. The multiplication-accumulation method for extremely low-precision training according to claim 1, characterized in that: The step of performing a specific number of dot products according to the bit width of each mantissa in the input data specifically includes: According to the bit width of each mantissa in the input data, the high-order multiplication is decomposed into a specific number of low-order multiplications, and after performing a specific number of dot multiplications on each of the low-order multiplications, the exponent in the dot product result is shifted to obtain a dot product result.

5. The multiplication-accumulation method for extremely low-precision training according to claim 4, characterized in that: Perform multiple dot products on the input data in a unified format, including the mantissa addition process and the normalization process; The mantissa addition process specifically includes: Calculate the sign, mantissa and exponent of the 4 products; Set the products with the same exponent and mantissa but opposite signs to 0; Use a parallel index comparison scheme to find the maximum exponent among the four products; Determine the initial shift amount of each product based on the exponent difference and the multiplication shape; Adjusting the offset of the maximum exponent to make it equal to the offset of the exponent in the dot product result; the offset is obtained according to the number of bits of the exponent and a preset fixed bias value; Based on the offset of the maximum exponent and the initial shift amount of the product corresponding to the maximum exponent, the final shift amount of the product corresponding to the maximum exponent is obtained, and according to the final shift amount and sign of the product corresponding to the maximum exponent, each product is aligned and the complement code is taken; The normalization process specifically includes: Perform addition tree reduction on the dot product result; Normalize the dot product result after the addition tree reduction.

6. The multiplication-accumulation method for extremely low-precision training according to claim 5, characterized in that: The step of adding the combined calculation result to the calculated partial sum by using the advance random rounding method comprises: In the mantissa addition process, a random number is filled into the empty space of the dot product result or the calculated part and the shift result, and the two are added to obtain an early random rounding carry; In the normalization process, the result of the early random rounding is corrected according to the number of leading zeros of the mantissa addition result.

7. The multiplication-accumulation method for extremely low-precision training according to claim 6, characterized in that: The step of adding the combined calculation result to the calculated partial sum by using the advance random rounding method specifically includes: Adding the dot product result to the calculated part and the addition result to obtain an addition result; An advance random rounding method is adopted to obtain an advance rounding carry; Normalizing the addition result for the second time to generate a sign, an exponent, and a mantissa before rounding; Based on the sign, exponent and mantissa before rounding and the lower 2 bits of the random number, the early rounding carry is corrected to obtain the final rounding carry.

8. A multiplication-accumulation operation device for extremely low-precision training, characterized in that: include: An acquisition module, used for acquiring the calculated partial sum and the data to be calculated, wherein the data to be calculated includes one or more of floating point, fixed point and logarithmic data formats; A decomposition module, used for decomposing the high-precision floating-point number into the sum of two low-precision floating-point numbers if there is a high-precision floating-point number in the data to be calculated; A decoding module, used for decoding the floating point, fixed point and logarithmic data formats in the data to be operated into a unified format of sign, exponent and mantissa, and generating input data in a unified format; The combined calculation module is used to perform multiple dot products on the input data in a unified format to achieve combined calculations of data formats with different precisions; The accumulation module is used for adding the combined calculation result to the calculated partial sum by adopting the advance random rounding method.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the multiplication-accumulation operation method for extremely low precision training is implemented as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multiplication-accumulation operation method for extremely low-precision training as described in any one of claims 1 to 7 is implemented.

Images