Multiplication and accumulation method and device for extremely low precision training

Abstract

The application provides a multiply-accumulate operation method and device for extremely low precision training, which comprises the following steps: obtaining an operated part and to-be-operated data, the to-be-operated data comprising one or more of floating point, fixed point and logarithmic data formats; if there is a high-precision floating point number in the to-be-operated data, the high-precision floating point number is decomposed into the sum of two low-precision floating point numbers; decoding the floating point, fixed point and logarithmic data formats in the to-be-operated data into a unified format of sign, exponent and mantissa to generate input data in the unified format; performing multiple point products on the input data in the unified format to realize combined calculation of different precision data formats; and using an early random rounding method to add the combined calculation result to the operated part. The application can support combined calculation among multiple formats, uses the early random rounding method to reduce the accumulator bit width under the premise of maintaining the precision requirement, and reduces the operation cost compared with the existing operation hardware, thereby improving the training efficiency under extremely low precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning hardware acceleration, and particularly relates to a multiply-accumulate operation method and device for extremely low-precision training. BACKGROUND

[0002] Deep learning has achieved extraordinary performance in various application fields, but the cost of computing resources, time and power consumption in training is too high, so researchers use quantized low-bit-width data to complete training, thereby significantly reducing memory occupation and saving data movement and processing cost. Recent research has proved the feasibility of ultra-low-precision (less than 8 bits) training, which can improve performance by 4-7 times compared with a 16-bit system, but requires selecting the optimal data format according to the distribution characteristics of the tensor, which introduces the need for multiple precision combination operations.

[0003] Random rounding is also necessary for extremely low-precision training, which can solve the problem of training stagnation and reduce the accumulator bit width while ensuring accuracy, thereby further saving resources. However, existing operation hardware generally does not support precision below 8 bits, multiple precision combination operations and random rounding, so the benefits of extremely low-precision training cannot be actualized, resulting in low training power consumption and area efficiency. SUMMARY

[0004] The present application provides a multiply-accumulate operation method and device for extremely low-precision training to solve the defect that the benefits of extremely low-precision training cannot be actualized in the prior art, resulting in low training power consumption and area efficiency, and to realize multiple precision combination multiply-accumulate operations for extremely low-precision training.

[0005] The present application provides a multiply-accumulate operation method for extremely low-precision training, comprising the following steps.

[0006] Obtain an already operated partial sum and data to be operated, wherein the data to be operated includes one or more of floating point, fixed point and logarithmic data formats;

[0007] If there is a high-precision floating point number in the data to be operated, decompose the high-precision floating point number into the sum of two low-precision floating point numbers;

[0008] Decode the floating point, fixed point and logarithmic data formats in the data to be operated into a unified format of sign, exponent and mantissa to generate input data in a unified format;

[0009] Perform multiple point products on the input data in a unified format to realize combination calculation of different precision data formats;

[0010] An early random rounding method is used to add the combination calculation result to the already operated partial sum.

[0011] The method for multiply-accumulate operation facing extremely low precision training provided by the application comprises the following steps:

[0012] decomposing a high-precision floating point number FH into two low-precision floating point numbers F1 and F2;

[0013] wherein in F1, the sign and the exponent are the same as those of FH, and the mantissa is the high-order mantissa of FH after rounding;

[0014] wherein in F2, the sign is the XOR of the sign of F1 and whether rounding is performed, the exponent is the number of leading 0s in the low-order mantissa of FH after complementation, and the mantissa is the low-order mantissa of FH after complementation.

[0015] The method for multiply-accumulate operation facing extremely low precision training provided by the application comprises the following steps of decoding floating point, fixed point and logarithmic data formats in the to-be-operated data into a unified format of sign, exponent and mantissa:

[0016] decoding floating point formats FP8 and FP6 into 1-bit sign, 5-bit exponent and 3-bit mantissa, and representing as S1E5M3;

[0017] decoding floating point format FP4 into 1-bit sign, 4-bit exponent and 1-bit mantissa, and representing as S1E4M1;

[0018] decoding logarithmic format LOG4 into 1-bit sign, 4-bit exponent and 1-bit mantissa, and representing as S1E4M1;

[0019] decoding fixed point format INT4 into 1-bit sign, 5-bit exponent and 3-bit mantissa, and representing as S1E5M3.

[0020] The method for multiply-accumulate operation facing extremely low precision training provided by the application comprises the following steps of performing point multiplication of a specific number of times according to the bit width of each mantissa in the input data:

[0021] According to the bit width of each mantissa in the input data, the high-order multiplication is decomposed into a specific number of low-order multiplications, and after performing point multiplication of the specific number of times on each low-order multiplication, the exponent in the point multiplication result is shifted to obtain a point multiplication result.

[0022] The method for multiply-accumulate operation facing extremely low precision training provided by the application performs multiple point multiplications on the input data in the unified format, including a mantissa addition process and a normalization process;

[0023] The mantissa addition process comprises the following steps:

[0024] calculating the sign, the mantissa and the exponent of the four products;

[0025] setting the product with the same exponent and mantissa but opposite sign to 0;

[0026] The maximum exponent of the four products is found using a parallel exponent comparison scheme;

[0027] An initial shift amount of each product is determined according to the exponent difference and the multiplication shape;

[0028] The offset of the maximum exponent is adjusted to be equal to the offset of the exponent in the dot product result; the offset is obtained according to the exponent bit number and a preset fixed bias value;

[0029] The final shift amount of the product corresponding to the maximum exponent is obtained based on the offset of the maximum exponent and the initial shift amount of the product corresponding to the maximum exponent, and each product is aligned and taken as a complement code according to the final shift amount of the product corresponding to the maximum exponent and a sign;

[0030] The normalization process specifically includes:

[0031] The dot product result is reduced by an addition tree;

[0032] The dot product result reduced by the addition tree is normalized.

[0033] According to the multiply-accumulate operation method for extremely low precision training provided by the application, the step of adding the combined calculation result and the already calculated partial sum includes:

[0034] In the mantissa addition process, a random number is filled into the empty position of the dot product result or the already calculated partial sum shift result, and the two are added to obtain an early random rounding carry;

[0035] In the normalization process, the result of the early random rounding is corrected according to the number of leading zeros of the mantissa addition result.

[0036] According to the multiply-accumulate operation method for extremely low precision training provided by the application, the step of adding the combined calculation result and the already calculated partial sum includes:

[0037] The dot product result and the addition result of the already calculated partial sum are added to obtain an addition result;

[0038] An early rounding carry is obtained by using the early random rounding method;

[0039] The addition result is normalized for a second time to generate a sign, an exponent and a mantissa before rounding;

[0040] Based on the sign, the exponent and the mantissa before rounding, and the low 2 bits of the random number, the early rounding carry is corrected to obtain a final rounding carry.

[0041] The application further provides a multiply-accumulate operation device for extremely low precision training, comprising the following modules.

[0042] An acquisition module is configured to acquire the operated partial sum and to-be-operated data, wherein the to-be-operated data comprises one or more of floating point, fixed point and logarithmic data formats;

[0043] A decomposition module is configured to decompose a high-precision floating point number into a sum of two low-precision floating point numbers if the to-be-operated data comprises the high-precision floating point number.

[0044] A decoding module is configured to decode the floating point, fixed point and logarithmic data formats in the to-be-operated data into a unified format of sign, exponent and mantissa, and to generate input data in the unified format.

[0045] A combination calculation module is configured to perform multiple point products on the input data in the unified format, and to realize combination calculation of different precision data formats.

[0046] An accumulation module is configured to add the combination calculation result to the operated partial sum by using an early random rounding method.

[0047] The application further provides an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the multiply-accumulate operation method for extremely low precision training according to any one of the above when executing the computer program.

[0048] The application further provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the multiply-accumulate operation method for extremely low precision training according to any one of the above.

[0049] The application further provides a computer program product comprising a computer program, wherein the computer program is executable on a processor to implement the multiply-accumulate operation method for extremely low precision training according to any one of the above.

[0050] The application provides a multiply-accumulate operation method and device for extremely low-precision training, and the method comprises the following steps: obtaining an operated part and to-be-operated data, wherein the to-be-operated data comprises one or more of floating point, fixed point and logarithmic data formats; if there is a high-precision floating point in the to-be-operated data, the high-precision floating point is decomposed into the sum of two low-precision floating points; the floating point, fixed point and logarithmic data formats in the to-be-operated data are decoded into a unified format of a sign, an exponent and a mantissa, to generate input data in the unified format; the input data in the unified format is subjected to multiple point multiplication, to realize combined calculation of different precision data formats; and the combined calculation result is added to the operated part by using an early random rounding method. The application can support combined calculation among multiple floating points, fixed points and logarithmic formats, and can reduce the accumulator bit width by using the early random rounding method under the premise of maintaining precision, so that high-precision support is realized at low cost, and the training efficiency under extremely low precision is improved compared with existing operation hardware. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the application or prior art, the following will briefly introduce the drawings needed in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort.

[0052] Figure 1 is one of the flowcharts of the multiply-accumulate operation method for extremely low-precision training provided by the application.

[0053] Figure 2 is the multi-precision combined multiply-accumulate operation architecture provided by the application.

[0054] Figure 3 is the multi-precision combined operation principle diagram provided by the application.

[0055] Figure 4 is the decoder logic diagram provided by the application.

[0056] Figure 5 is the multiple point multiplication unit structure diagram provided by the application.

[0057] Figure 6 is the early random rounding principle diagram of the bit width unequal addition provided by the application.

[0058] Figure 7 is the high-precision decomposition principle diagram provided by the application.

[0059] Figure 8 is the structure diagram of the multiply-accumulate operation device for extremely low-precision training provided by the application.

[0060] Figure 9 is a structural schematic diagram of an electronic device provided by the present application. DETAILED DESCRIPTION

[0061] To make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in conjunction with the drawings in the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort belong to the scope of protection of the present application.

[0062] The present application will be described in detail below in conjunction with the drawings in the specification. The specific operation methods in the method embodiments can also be applied to the device embodiments or the system embodiments. In the description of the present application, unless otherwise specified, "at least one" includes one or more. "Multiple" refers to two or more. For example, at least one of A, B and C includes: A alone, B alone, A and B together, A and C together, B and C together, and A, B and C together. In the present application, " / " means or, for example, A / B can mean A or B; "and / or" in this document only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone.

[0063] The present application will be described in detail below in conjunction with the specific embodiments.

[0064] In some specific embodiments of the present application, as shown in Figure 1 The present application provides a multiply-accumulate operation method for very low precision training, which includes:

[0065] Step 110, obtaining an operated part and and to-be-operated data, the to-be-operated data including one or more of floating point, fixed point and logarithmic data formats;

[0066] Step 120, if there is a high-precision floating point number in the to-be-operated data, decomposing the high-precision floating point number into the sum of two low-precision floating point numbers;

[0067] Step 130, decoding the floating point, fixed point and logarithmic data formats in the to-be-operated data into a unified format of sign, exponent and mantissa, to generate input data in a unified format;

[0068] Step 140, performing multiple point products on the input data in a unified format to realize combined calculation of different precision data formats;

[0069] Step 150, using the early random rounding method to add the combined calculation result to the operated part and.

[0070] It should be noted that the existing multiply-accumulate operation hardware scheme cannot realize the operation of multiple precision combinations below 8-bit precision, and cannot train through random rounding at extremely low precision, so it is difficult to actualize the benefits of extremely low precision training, and cannot meet the demand of low cost and high efficiency of extremely low precision training.

[0071] Therefore, the present application decomposes a high-precision floating-point number into the sum of two low-precision floating-point numbers, decodes the floating-point, fixed-point, and logarithmic data formats in the to-be-operated data into a unified format of sign, exponent, and mantissa, and then performs multiple point products on the input data in the unified format to realize combined calculation of different precision data formats; and adopts an early random rounding method to add the combined calculation result to the operated part sum, thereby reducing the accumulator bit width under the premise of maintaining the precision requirement and improving the training efficiency at extremely low precision.

[0072] In some possible embodiments of the present application, the step of decomposing a high-precision floating-point number into the sum of two low-precision floating-point numbers specifically comprises:

[0073] decomposing the high-precision floating-point number FH into two low-precision floating-point numbers F1 and F2;

[0074] In F1, the sign and the exponent are the same as those of FH, and the mantissa is the high-order mantissa of FH after rounding.

[0075] In F2, the sign is the XOR of the sign of F1 and whether rounding is performed, the exponent is the number of leading 0s in the low-order mantissa of FH after complementation, and the mantissa is the low-order mantissa of FH after complementation.

[0076] Specifically, the embodiment provides an implementation of decomposing a high-precision floating-point number into the sum of two low-precision floating-point numbers. By decomposing a high-precision floating-point number into the sum of two low-precision floating-point numbers, the decomposition results are sequentially input into a low-precision calculation unit, and a high-precision approximate calculation result is obtained after multiple period accumulations, thereby improving the precision range supported by the design at a small area cost.

[0077] In a possible embodiment, in step 120, the high-precision floating-point number FH is decomposed into the sum of two low-precision floating-point numbers F1 and F2, specifically, the sign and the exponent of F1 are the same as those of FH, and the mantissa is the high-order mantissa of FH after rounding, the sign of F2 is the XOR of the sign of F1 and whether rounding is performed, the exponent is the number of leading 0s in the low-order mantissa of FH after complementation, and the mantissa is the low-order mantissa of FH after complementation.

[0078] In some possible embodiments of the present application, the step of decoding the floating-point, fixed-point, and logarithmic data formats in the to-be-operated data into a unified format of sign, exponent, and mantissa specifically comprises:

[0079] decode the floating point format FP8 and FP6 into 1-bit sign, 5-bit exponent and 3-bit mantissa, denoted as S1E5M3;

[0080] decode the floating point format FP4 into 1-bit sign, 4-bit exponent and 1-bit mantissa, denoted as S1E4M1;

[0081] decode the logarithm format LOG4 into 1-bit sign, 4-bit exponent and 1-bit mantissa, denoted as S1E4M1;

[0082] decode the fixed point format INT4 into 1-bit sign, 5-bit exponent and 3-bit mantissa, denoted as S1E5M3.

[0083] Specifically, the embodiment provides an implementation of decoding the floating point, fixed point and logarithm data format in the to-be-operated data into a unified format of sign, exponent and mantissa, and decoding the floating point, fixed point and logarithm format into two unified formats of sign, exponent and mantissa: S1E5M3 (1-bit sign, 5-bit exponent and 3-bit mantissa) or S1E4M1, so as to facilitate the operation unit to perform operation in the unified format.

[0084] Specifically, in step 130, the process of decoding the floating point, fixed point and logarithm data format in the to-be-operated data into a unified format of sign, exponent and mantissa to generate input data in the unified format is specifically: the floating point format FP8 and FP6 are decoded into S1E5M3 (1-bit sign, 5-bit exponent and 3-bit mantissa), the floating point format FP4 is decoded into S1E4M1, the logarithm format LOG4 can be regarded as E4M0 and is decoded into S1E4M1, and the fixed point format INT4 can be regarded as a floating point number with an exponent of 0 and is decoded into S1E5M3.

[0085] In some possible embodiments of the present application, the step of performing a certain number of point multiplications according to the bit width of each mantissa in the input data specifically includes:

[0086] According to the bit width of each mantissa in the input data, the high-bit multiplication is decomposed into a certain number of low-bit multiplications, a certain number of point multiplications are performed on each low-bit multiplication, and then the exponent in the point multiplication result is shifted to obtain a point multiplication result.

[0087] Specifically, the embodiment provides an implementation of performing a certain number of point multiplications according to the bit width of each mantissa in the input data, and the number of low-bit multiplications obtained by decomposition is determined according to the bit width of the mantissa, and the number determines the number of point multiplications performed.

[0088] In a possible implementation, the dot product is performed 1 / 2 / 4 times according to the mantissa bit width to complete the combined calculation of LOG4, FP4(E3M0,E2M1), FP6(E3M2, E2M3), FP8(E5M2, E4M3) and INT4, and the results are accumulated in FP12 or INT8 format.

[0089] The process of performing 1 / 2 / 4 dot products based on the mantissa bit width is as follows: decompose the 4-bit multiplication into 4 2-bit multiplications, then shift the product left by 4 / 2 / 2 / 0 bits and reduce it; changing the shift amount to 2 / 2 / 0 / 0 bits or 0 / 0 / 0 / 0 bits can achieve 2 4×2 bit multiplications or 4 2×2 bit multiplications.

[0090] In step 140, this embodiment of the invention extends the above idea to mixed operations of multiple precisions such as floating-point, fixed-point, and logarithmic, and performs multiple dot products on input data of the same format to achieve combined calculations of data formats with different precisions.

[0091] Furthermore, multiple dot products are completed based on the input in a unified format, specifically including 5 pipeline stages; the first 3 stages obtain 1 / 2 / 4 term dot product results in intermediate format, namely product decoding and cancellation, multiplication and exponent comparison and order shift, addition tree reduction and product normalization; the last 2 stages obtain the final result with the addition of partial sums, namely the addition of intermediate dot product results and partial sums, and the normalization and early random rounding of addition results.

[0092] In some possible embodiments of the present invention, multiple dot products are performed on input data of a uniform format, including a mantissa addition process and a normalization process.

[0093] The mantissa addition process specifically includes:

[0094] Calculate the sign, mantissa, and exponent of four products;

[0095] Set the product of exponents and mantissas that are the same but have opposite signs to 0;

[0096] The largest exponent among the four products is obtained using a parallel exponent comparison scheme.

[0097] The initial shift amount of each product is determined based on the exponential difference and the shape of the multiplication.

[0098] Adjust the offset of the maximum exponent to make it equal to the offset of the exponent in the dot product result; the offset is obtained based on the number of exponent bits and a preset fixed offset value;

[0099] Based on the offset of the maximum exponent and the initial shift of the product corresponding to the maximum exponent, the final shift of the product corresponding to the maximum exponent is obtained. Based on the final shift of the product corresponding to the maximum exponent and the sign, the products are aligned and then their complements are taken.

[0100] The normalization process specifically includes:

[0101] The dot product result is subjected to an addition tree reduction;

[0102] The dot product result subjected to the addition tree reduction is normalized.

[0103] Specifically, the embodiment provides an implementation of a multi-term dot product of input data in a unified format, which divides the operation of the multi-term dot product into a mantissa addition and a normalization process, and finally obtains the dot product result through the two processes.

[0104] In possible embodiments, the signs of the four products are obtained by XOR of the input signs, and the exponents of the four products are obtained by adding the exponents of the input multipliers.

[0105] It should be noted that the mantissa addition process and the normalization process in the embodiment can be regarded as operations performed by the first three pipeline stages, specifically, the first three pipeline stages obtain the dot product result in an intermediate format, specifically, first, the signs, mantissas and exponents of the four products are calculated, the signs of the four products can be obtained by XOR of the input signs, and the exponents of the four products can be obtained by adding the exponents of the input multipliers. Then, the products with the same exponents and mantissas and opposite signs are set to 0, secondly, the maximum exponent among the four products is compared in parallel, and the shift amount of each product is determined according to the exponent difference and the multiplication shape and is complemented, and finally, the dot product result subjected to the addition tree reduction is normalized.

[0106] Specifically, if the current mode is to calculate the sum of four FP4xFP4, each product represents an FP4xFP4; if the current mode is to calculate one FP8xFP8, each product represents 1 / 4 of FP8xFP8, and each product is shifted and then reduced to generate an FP8xFP8 result.

[0107] In some possible embodiments of the application, an early random rounding method is used to add the combination calculation result to the operated partial sum, which includes:

[0108] In the mantissa addition process, random numbers are filled into the empty positions of the dot product result or the shifted result of the operated partial sum, and the two are added to obtain an early random rounding carry;

[0109] In the normalization process, the result of the early random rounding is corrected according to the number of leading zeros of the mantissa addition result.

[0110] Specifically, the embodiment provides an implementation of adding the combination calculation result to the operated partial sum, which uses an early random rounding method for addition of unequal bit widths in the accumulation stage, avoids the training stagnation phenomenon, and can reduce the accumulator bit width under the premise of ensuring accuracy, to further save resources.

[0111] It should be noted that the process of adding the combination calculation result to the partial sum in the embodiment can be regarded as the operation performed by the last two pipeline stages, and the last two pipeline stages obtain the final result of the partial sum. Specifically, the partial sum is first added to the dot product result after being decoded, compared in the exponent, and aligned, and the early random rounding is started at the same time. Then, the addition result is normalized, and the early random rounding result is corrected to obtain the final rounding carry.

[0112] In some possible embodiments of the present application, the step of adding the combination calculation result to the partial sum by using the early random rounding method specifically includes:

[0113] adding the dot product result to the addition result of the partial sum to obtain an addition result;

[0114] obtaining an early rounding carry by using the early random rounding method;

[0115] normalizing the addition result for a second time to generate a pre-rounding sign, exponent, and mantissa;

[0116] correcting the early rounding carry based on the pre-rounding sign, exponent, and mantissa, and the low 2 bits of the random number to obtain a final rounding carry.

[0117] Specifically, the embodiment provides a specific implementation of adding the combination calculation result to the partial sum. The early rounding carry is obtained by using the early random rounding method, and the final rounding carry is obtained by correcting the early rounding carry based on the pre-rounding sign, exponent, and mantissa.

[0118] Specifically, in step 150, the process of adding the combination calculation result to the partial sum uses the early random rounding method for the addition of bit widths that are not equal. Specifically, in the mantissa addition process, the random number is filled into the empty bit of the dot product or the partial sum shift result, and the early random rounding carry is obtained by adding the two results. In the normalization stage, the result of the early random rounding is corrected according to the number of leading 0s of the mantissa addition result.

[0119] The processes of the above embodiments will be described in detail through specific embodiments.

[0120] In possible embodiments, Figure 2 is a multi-precision combined multiply-accumulate operation architecture provided by the present application, as shown in Figure 2 The multi-precision combined multiply-accumulate operation architecture is composed of four parts, including a high-precision analog controller, a left / right decoder, a multi-item dot product unit, and a random number generator.

[0121] Specifically, the high-precision analog controller: decomposes the input FP12 into the sum of two FP8, and simulates high-precision calculation through the accumulation of multiple low-precision calculations; the left / right decoder: decodes the floating point, fixed point, and logarithmic format into the unified format of sign, exponent, and mantissa; the multiple point product unit: completes the multiple point product in the unified format, and the output format is INT8 when the input is INT4, otherwise the output format is FP12; the random number generator: uses a linear feedback shift register (LSFR) to generate a 12-bit random number for random rounding.

[0122] That is, in step 120, if there is a high-precision floating point number in the data to be operated, the process of decomposing the high-precision floating point number into the sum of two low-precision floating point numbers can be realized by the high-precision analog controller in Figure 2 ; in step 130, the process of decoding the floating point, fixed point, and logarithmic data format in the data to be operated into the unified format of sign, exponent, and mantissa to generate the input data in the unified format can be realized by the left / right decoder in Figure 2 ; in step 140, the process of performing multiple point products on the input data in the unified format to realize the combined calculation of different precision data formats can be realized by the multiple point product unit in Figure 2 ; in step 150, the process of adding the combined calculation result to the operated partial sum using the early random rounding method can be realized with the assistance of the random number generator in Figure 2 .

[0123] Under this architecture, according to the mantissa bit width of the input data, after completing 1 / 2 / 4 point products each time, the floating point format is accumulated at FP12, and the fixed point format is accumulated at INT8, as shown in the following formula,

[0124] (1)

[0125] wherein F is the operation result, is the operated partial sum, , is the data to be operated, and K is the number of point products performed in each operation.

[0126] In possible embodiments, as shown in Table 1, Table 1 is a precision combination type table supported by the embodiments of the present application, which shows the precision modes supported by the architecture.

[0127] Table 1

[0128]

[0129] Take K = 1 / 3, precision combination FP12xFP12 as an example, one FP12 is decomposed into the sum of two FP8, therefore, once FP12xFP12 can be decomposed into 4 times of FP8xFP8 accumulated on FP12, namely:

[0130] FP12 A x FP12 B = FP8 AH x FP8 BH + FP8 AH x FP8 BL + FP8 AL x FP8 BH + FP8 AL x FP8 BL ;

[0131] Among them, FP8 AL x FP8 BL has little effect on the result, and can be omitted, therefore, three times of FP8 dot product are needed to obtain once FP12xFP12, K = 1 / 3. The calculation of FP12xFP8 is the same, but only two times of multiplication accumulation is needed. The high-precision analog controller realizes FP12 decomposition, and inputs the decomposition result to the low-precision multiple-term dot product unit in sequence to complete accumulation, so as to realize high-precision analog.

[0132] In possible embodiments, Figure 3 is the multi-precision combination operation principle diagram provided by the application, as shown in Figure 3 Any 2 n bit fixed-point multiplication is decomposed into the sum of multiple 2-bit multiplications after shifting. For example, to realize 4-bit multiplication, the input is split into high 2 bits and low 2 bits, then 4 2-bit multiplications are performed, the products are left shifted by 4 / 2 / 2 / 0 bits, and finally an 8-bit addition tree is used for reduction. Similarly, the above circuit can also realize 2 4x2 multiplications or 4 2x2 multiplications, only the shift amount needs to be changed to 2 / 2 / 0 / 0 and 0 / 0 / 0 / 0.

[0133] The scheme of the above embodiments can be extended to floating-point operations. Since FP8 contains at most 4 bits of mantissa, and FP4 / LOG4 contains at most 2 bits of mantissa, Figure 3 the variant of the circuit in can realize once FP8xFP8 or twice FP8xFP4 or four times FP8xFP4. The required changes include: the product is complemented according to the sign, shifted according to the exponent difference, and decoded, normalized and rounded, etc.

[0134] Figure 4 In possible embodiments, Figure 4As shown, the decoder decodes multiple basic types into sign, exponent and mantissa form, where FP4 is decoded into E4M1 to be compatible with LOG4(E4M0) and E2M1. FP8 is decoded into E5M3 to be compatible with E5M2 and E4M3. Figure 4 Taking the floating point decoding as an example, the fixed point is regarded as a floating point number with an exponent of 0.

[0135] Specifically, the 16-bit input of the decoder is composed of 4 FP4 or 2 FP8, and in some cases, all the data does not need to be used, such as Figure 4 (a) shown. The result of the sign decoding is the sign of the 4 products, such as Figure 4 (b) shown. When decoding the exponent, the bias of the exponent needs to be adjusted, such as when E3M0 (bias = 3) is decoded into E4M1 (bias = 7), the exponent needs to be added by 4. Part of the FP8 exponent decoding result is invalid and is set to 0, such as Figure 4 (c) shown. The 8-bit result of the mantissa decoding contains the leading bits, and in order to realize the 4-bit multiplication required by FP8, we decompose the 4-bit mantissa into 4 2-bit mantissas according to the logic of Figure 3 , and then use different permutation methods in the left / right decoder, such as Figure 4 (d) shown.

[0136] In possible embodiments, Figure 5 is a structure diagram of a multiple dot product unit provided by the present application, as shown in Figure 5 The multiple dot product unit includes 5 pipeline stages, the first 3 stages obtain 1 / 2 / 4 dot product results in an intermediate format, and the last 2 stages add the intermediate results to the partial sum to obtain the final result. The fixed point and floating point operations share all the calculation logic, which is not shown in the figure.

[0137] The first stage pipeline completes product decoding and cancellation, specifically: first, calculate the sign, mantissa and exponent of the 4 products, get the sign of the 4 products through the exclusive or of the input signs, and add the exponents of the input multipliers to get the 4 product exponents. Then, set the products with the same exponent and opposite signs to 0, so as to avoid the precision loss caused by the right shift of the small product due to the two opposite large products.

[0138] The second stage pipeline completes multiplication, exponent comparison and alignment shift, specifically: first, use a parallel exponent comparison scheme to get the maximum exponent of the 4 products, and then determine the shift amount according to the exponent difference and the multiplication shape. Then adjust the offset of the maximum exponent to make it equal to the offset of the result exponent. Finally, according to the shift amount and the sign, take the complement code after aligning the products.

[0139] The third stage pipeline completes the normalization of the dot product result and the addition of the partial sum, specifically: the dot product result after the addition tree is normalized to facilitate subsequent standard floating-point addition. The high 9 bits of the normalized mantissa result are used for floating-point addition, and the low 5 bits are used for early random rounding (SR).

[0140] The fourth stage pipeline completes the addition of the dot product result and the partial sum, specifically: the partial sum is added to the dot product result after decoding, exponent comparison, and alignment. The early SR is performed in parallel with the addition to generate an early rounding carry. The high 10 bits of the addition result are used for normalization left shift, and the low 6 are used for SR correction.

[0141] The fifth stage pipeline completes the normalization and rounding of the addition result, specifically: the addition result is normalized for the second time to generate the sign, exponent, and mantissa before rounding. The SR correction is obtained through the normalized result, the low 2 bits of the random number, and the early rounding carry. The final rounding carry is obtained. Non-standard numbers are supported in the rounding stage.

[0142] It is worth noting that traditional stochastic rounding (SR) is performed after normalization, resulting in a large delay. To optimize the delay, (Sami Ben Ali et al. 2024. A Stochastic Rounding-Enabled Low-Precision FloatingPoint MAC for DNN Training. In 2024 Design, Automation Test in Europe Conference Exhibition (DATE). 1-6.) proposed an Eager-SR strategy that performs SR before normalization and corrects SR during normalization, but this method is only applicable to floating-point addition with equal addend bit widths.

[0143] Considering that the bit widths of the dot product result (PROD) and the partial sum (PSUM) in the multiply-accumulate operation are usually not equal, the present application proposes an early SR strategy (collectively referred to as Variable-SR) for bit width unequal addition, as shown in Figure 5 The early SR optimizes the delay and reduces the normalization bit width, achieving performance improvement.

[0144] In possible embodiments, Figure 6 is the early random rounding schematic diagram for bit width unequal addition provided by the present application, as shown in Figure 6 Figure 6 ​(a) shows how Variable-SR strategy performs early SR in mantissa addition, specifically, after PROD or PSUM alignment, 10-bit random numbers are padded into the shifted result, the high 8 bits of the shifted result are used for mantissa addition, and the low 10 bits are used for early SR addition, both are completed in parallel. When PROD is aligned, random numbers are padded to the right of PSUM LSB, and SR addition is performed with the low bits of PROD. When PSUM is aligned, the low 5 bits of random numbers are padded to the right of PROD LSB, first SR addition is performed, and then the addition result is added with the high 5 bits of random numbers to get early SR rounding carry (S1S2).

[0145] Further, Figure 6 (b) shows how Variable-SR strategy performs SR correction in normalization, specifically, according to the number of leading zeros cnt of mantissa addition result, SR correction is performed in the following 4 cases:

[0146] 1) cnt=0: mantissa addition produces carry, the low 2 bits of the addition result, 2-bit random numbers, and S1S2 are added to get the final rounding carry (C1C2).

[0147] 2) cnt=1: mantissa addition does not produce carry, the LSB of the addition result, 1-bit random numbers, and S1S2 are added to get C1C2.

[0148] 3) cnt=2: mantissa addition produces 1-bit borrow, C1C2 is equal to S1S2.

[0149] 4) cnt≥3: mantissa addition produces cancellation, the addition result is accurate, and no SR is needed. According to the difference between the exponents of PROD and PSUM, the low 5 / 6 bits of PROD are put into the low bits of the normalization shifter to ensure that the precision is lossless.

[0150] In possible embodiments, Figure 7 is a high-precision decomposition schematic provided by the present application, as shown in Figure 7 , an FP12 is decomposed into the sum of two FP8s, i.e. FP12=FP8 H +FP8 L Therefore, one FP12xFP12 can be decomposed into four FP8xFP8 on FP12, i.e.:

[0151] FP12 A xFP12 B =FP8 AH xFP8 BH +FP8 AH xFP8 BL +FP8 AL xFP8BH +FP8 AL ×FP8 BL ,

[0152] where FP8 AL ×FP8 BL has little effect on the result, and is omitted. The same applies to FP12 × FP8, but only two multiply-accumulates are needed. A high-precision analog controller implements the FP12 decomposition and feeds the results sequentially to the low-precision multiply-accumulate units for accumulation to achieve high-precision analog.

[0153] The specific decomposition scheme is: first make the sign and exponent of FP8 H the same as FP12, and then round FP8 H to the nearest according to the third bit R of the FP12 mantissa. The exponent of FP8 L is adjusted so that it can represent the low-order mantissa of FP12. When R = 0, the sign is the same as FP12, and the last 4 bits of FP12 are counted with leading zeros (LZC), and the leading 1 is used as the implicit bit of FP8 L , and the next 2 bits are used as the mantissa. When R = 1, the sign is opposite to FP12, and the last 4 bits of FP12 are complemented and then LZC is performed.

[0154] The above decomposition is completely accurate when the FP12 exponent is greater than or equal to 7, otherwise it cannot be used because the theoretical FP8 L exponent may be less than 1. We reduce the minimum FP12 exponent that can be accurately decomposed from 7 to 5 by supporting non-standard FP8 L representation.

[0155] The multiply-accumulate operation method for extremely low-precision training provided by the embodiments of the present application supports mutual calculation between multiple floating points, fixed points and logarithmic formats, decomposes high-precision floating points into the sum of two low-precision floating points, simulates high-precision calculation through multiple low-precision calculations, uses the early random rounding method to reduce the accumulator bit width while maintaining the accuracy requirement, and designs a high-precision analog controller to support high-precision at low cost, thereby improving the training efficiency under extremely low precision compared with existing operation hardware.

[0156] The multiply-accumulate operation device for extremely low-precision training provided by the embodiments of the present application is described below. The multiply-accumulate operation device for extremely low-precision training described below can be referred to in conjunction with the multiply-accumulate operation method for extremely low-precision training described above.

[0157] In some specific embodiments of the present application, as shown in Figure 8 , the scheme provides a multiply-accumulate operation device for extremely low-precision training, comprising:

[0158] The acquisition module 81 is configured to acquire the operated partial sum and to-be-operated data, wherein the to-be-operated data includes one or more of floating point, fixed point and logarithmic data formats.

[0159] The decomposition module 82 is configured to decompose a high-precision floating point number into a sum of two low-precision floating point numbers if the to-be-operated data includes the high-precision floating point number.

[0160] The decoding module 83 is configured to decode the floating point, fixed point and logarithmic data formats in the to-be-operated data into a unified format of sign, exponent and mantissa, to generate input data in the unified format.

[0161] The combination calculation module 84 is configured to perform multiple point products on the input data in the unified format, to realize combination calculation of different precision data formats.

[0162] The accumulation module 85 is configured to add the combination calculation result to the operated partial sum by using an early random rounding method.

[0163] The multiplication-accumulation operation device for extremely low-precision training provided in the embodiments has similar implementation principles and beneficial effects to those of the multiplication-accumulation operation method for extremely low-precision training described above. For details, refer to the implementation principles and beneficial effects of the multiplication-accumulation operation method for extremely low-precision training described above, which will not be described here again.

[0164] Figure 9 An example of a schematic diagram of a physical structure of an electronic device is shown in FIG. 10. Figure 9 As shown in FIG. 10, the electronic device can include a processor 910, a communications interface 920, a memory 930 and a communications bus 940, wherein the processor 910, the communications interface 920 and the memory 930 can complete communication with each other through the communications bus 940. The processor 910 can invoke a logical instruction in the memory 930 to execute the multiplication-accumulation operation method for extremely low-precision training, which includes the following steps: acquiring an operated partial sum and to-be-operated data, wherein the to-be-operated data includes one or more of floating point, fixed point and logarithmic data formats; decomposing a high-precision floating point number into a sum of two low-precision floating point numbers if the to-be-operated data includes the high-precision floating point number; decoding the floating point, fixed point and logarithmic data formats in the to-be-operated data into a unified format of sign, exponent and mantissa, to generate input data in the unified format; performing multiple point products on the input data in the unified format, to realize combination calculation of different precision data formats; and adding the combination calculation result to the operated partial sum by using an early random rounding method.

[0165] Furthermore, the logical instructions in the aforementioned memory 930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0166] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the multiply-accumulate operation method for extremely low-precision training provided by the above methods. The method includes: acquiring the already calculated partial sum and the data to be calculated, wherein the data to be calculated includes one or more of floating-point, fixed-point, and logarithmic data formats; if there are high-precision floating-point numbers in the data to be calculated, decomposing the high-precision floating-point numbers into the sum of two low-precision floating-point numbers; decoding the floating-point, fixed-point, and logarithmic data formats in the data to be calculated into a unified format of sign, exponent, and mantissa, generating input data of a unified format; performing a multinomial dot product on the input data of the unified format to realize the combined calculation of different precision data formats; and adding the combined calculation result to the already calculated partial sum using an advance random rounding method.

[0167] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a multiply-accumulate operation method for extremely low-precision training provided by the methods described above. The method includes: acquiring the already calculated partial sum and the data to be calculated, wherein the data to be calculated includes one or more of floating-point, fixed-point, and logarithmic data formats; if the data to be calculated contains a high-precision floating-point number, decomposing the high-precision floating-point number into the sum of two low-precision floating-point numbers; decoding the floating-point, fixed-point, and logarithmic data formats in the data to be calculated into a unified format of sign, exponent, and mantissa, generating input data of a unified format; performing a multinomial dot product on the input data of the unified format to achieve combined calculation of different precision data formats; and adding the combined calculation result to the already calculated partial sum using an advance random rounding method.

[0168] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0169] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0170] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for multiply-accumulate operation oriented to extremely low precision training, characterized in that, The method comprises the following steps: acquiring an operated partial sum and to-be-operated data, the to-be-operated data comprising one or more of floating-point, fixed-point and logarithmic data formats; if there is a high-precision floating-point number in the to-be-operated data, decomposing the high-precision floating-point number into the sum of two low-precision floating-point numbers; decoding the floating-point, fixed-point and logarithmic data formats in the to-be-operated data into a unified format of sign, exponent and mantissa to generate input data in the unified format; performing multiple point multiplication on the input data in the unified format to realize combined calculation of different precision data formats; using an advanced random rounding method to add the combined calculation result to the operated partial sum; the step of performing multiple point multiplication on the input data in the unified format to realize combined calculation of different precision data formats specifically comprises five pipeline stages: the first stage of pipeline completes product decoding and cancellation, the second stage of pipeline completes multiplication, exponent comparison and alignment shift, the third stage of pipeline completes reduction and point multiplication result normalization; the fourth stage of pipeline completes addition of the point multiplication result and the partial sum, and the fifth stage of pipeline completes normalization and rounding of the addition result.

2. The method according to claim 1, characterized in that, the step of decomposing the high-precision floating-point number into the sum of two low-precision floating-point numbers specifically comprises the following steps: decomposing the high-precision floating-point number FH into two low-precision floating-point numbers F1 and F2; in F1, the sign and the exponent are the same as those of FH, and the mantissa is the high-order mantissa of FH after rounding; in F2, the sign is the XOR of the sign of F1 and whether rounding is performed, the exponent is the number of leading 0s in the low-order mantissa after FH exponent subtraction and complementation, and the mantissa is the low-order mantissa after FH exponent subtraction and complementation.

3. The method of claim 1, wherein, the step of decoding the floating-point, fixed-point and logarithmic data formats in the to-be-operated data into a unified format of sign, exponent and mantissa specifically comprises the following steps: decoding floating-point formats FP8 and FP6 into 1-bit sign, 5-bit exponent and 3-bit mantissa, represented as S1E5M3; decoding floating-point format FP4 into 1-bit sign, 4-bit exponent and 1-bit mantissa, represented as S1E4M1; decoding logarithmic format LOG4 into 1-bit sign, 4-bit exponent and 1-bit mantissa, represented as S1E4M1; decoding fixed-point format INT4 into 1-bit sign, 5-bit exponent and 3-bit mantissa, represented as S1E5M3.

4. The method of Claim 1, wherein, the step of performing multiple point multiplication on the input data in the unified format specifically comprises the following steps: according to the bit width of each mantissa in the input data, decomposing high-order multiplication into a specific number of low-order multiplications, performing a specific number of point multiplications on each low-order multiplication, and then shifting the exponent in the point multiplication result to obtain a point multiplication result.

5. The method of claim 4, wherein, performing multiple point multiplication on the input data in the unified format comprises a mantissa addition process and a normalization process; the mantissa addition process specifically comprises the following steps: calculating the sign, mantissa and exponent of the four products generated by multiple point multiplication; setting the product with the same exponent and mantissa but opposite sign to 0; using a parallel exponent comparison scheme to obtain the maximum exponent among the four products; determining the initial shift amount of each product according to the exponent difference and the multiplication shape; adjusting the offset of the maximum exponent to be equal to the offset of the exponent in the point multiplication result; the offset is obtained according to the exponent bit number and a preset fixed offset value; Based on the maximum exponent offset and the initial shift amount of the maximum exponent corresponding product, a final shift amount of the maximum exponent corresponding product is obtained, and each product is aligned according to the final shift amount of the maximum exponent corresponding product and the sign, and then the complement code is taken; The normalization process specifically includes: The dot product result is subjected to addition tree reduction; The dot product result subjected to addition tree reduction is normalized.

6. The method of claim 5, wherein, The step of adding the combination calculation result to the already operated partial sum by using the early random rounding method includes: In the mantissa addition process, a random number is filled into the empty position of the dot product result or the already operated partial sum shift result, and the dot product result is added to the already operated partial sum to obtain an early random rounding carry; The early random rounding carry is subjected to a second normalization to generate the sign, the exponent and the mantissa before rounding; Based on the sign, the exponent and the mantissa before rounding, the low 2 bits of the random number, the early random rounding carry is corrected to obtain a final rounding carry.

7. A multiply-accumulate operation device oriented to very low precision training, characterized by, It includes: An acquisition module is configured to acquire an already operated partial sum and to-be-operated data, the to-be-operated data including one or more of floating point, fixed point and logarithmic data formats; A decomposition module is configured to decompose a high-precision floating point number into the sum of two low-precision floating point numbers if the to-be-operated data includes the high-precision floating point number; A decoding module is configured to decode the floating point, fixed point and logarithmic data formats in the to-be-operated data into a unified format of a sign, an exponent and a mantissa to generate input data in the unified format; A combination calculation module is configured to perform multiple dot products on the input data in the unified format to realize combination calculation of different precision data formats; An accumulation module is configured to add the combination calculation result to the already operated partial sum by using an early random rounding method. The step of performing multiple dot products on the input data in the unified format to realize combination calculation of different precision data formats specifically includes five pipeline stages: the first stage of pipeline completes product decoding and offset, the second stage of pipeline completes multiplication, exponent comparison and alignment shift, the third stage of pipeline completes reduction and normalization of the dot product result; the fourth stage of pipeline completes addition of the dot product result to the partial sum, and the fifth stage of pipeline completes normalization and rounding of the addition result.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the computer program to realize the multiply-accumulate operation method facing extremely low precision training according to any one of claims 1 to 6. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the multiply-accumulate operation method facing extremely low precision training according to any one of claims 1 to 6.

Images