A floating-point multiply-accumulate unit with a fused precision conversion function and its application method

Abstract

The present invention discloses a floating-point multiply-add unit with a fused precision conversion function and an application method thereof. By modifying the exponent adjustment module, mantissa rounding module, and result output module of the standard floating-point multiply-add operation unit, the present invention enables the exponent adjustment module to perform exponent adjustment based on the number of leading zeros in the mantissa before normalization shift, the difference between the current mixed-precision operation addend precision, and the exponent base value of the target precision; the mantissa rounding module is used to round the mantissa according to the type of target precision to be supported; the result output module is used to combine the sign bit, exponent, and rounded mantissa, and a combined result is obtained for each precision, and one of the combined results is selected as the final output result according to the specific precision conversion control signal. By modifying the standard floating-point multiply-add operation unit to fuse the precision conversion function, the present invention can reduce the hardware overhead and improve the implementation efficiency of deep learning algorithms, and has the advantages of low overhead and high efficiency.

Description

Technical Field

[0001] The present invention belongs to the design technology of floating-point functional units in the field of processor design technology, and particularly relates to a floating-point multiply-add unit with a fused precision conversion function and an application method thereof. Background Art

[0002] In current deep learning hardware platforms, algorithm model training often uses floating-point data formats for calculation. The data stream is relatively complex and involves various types of operations, resulting in a large hardware overhead. To reduce the huge hardware overhead during training, current mainstream deep learning training platforms often adopt a mixed-precision training method. By using low-precision multiplication, the hardware throughput is increased while the hardware complexity is reduced, and high-precision addition is supplemented to maintain the training accuracy. However, during the mixed-precision training process of deep learning network models, since the data precision requirements of each operator in the model are different, the data needs to be frequently converted in precision during the transmission process between each network layer to reduce overflow during the calculation process. This poses new requirements for the hardware calculation mode. To meet the data precision conversion requirements during the training process of deep learning models, traditional methods often design dedicated precision conversion units and design independent conversion circuits for each type of precision conversion. The high-precision result output by the mixed-precision floating-point multiply-add unit using low-precision multiplication and high-precision addition is sent to the precision conversion unit, and according to the precision requirements of the next network layer, the high-precision multiply-add result is converted into the target (low) precision output. The defect of the traditional implementation method is that the two operations of floating-point multiply-add and precision conversion are completely independent and implemented in different hardware units, resulting in a large overhead. Summary of the Invention

[0003] Analysis of the actual deep learning algorithm mode reveals that: the two operators of floating-point multiply-add and precision conversion are usually adjacent and can be fused. The essence of floating-point precision conversion is to perform corresponding processing on the exponent and mantissa of the floating-point number, which is similar to the processing methods of exponent adjustment and mantissa rounding in floating-point multiply-add operations. If the two are fused, the hardware overhead can be effectively reduced, and at the same time, the software control difficulty can be reduced. Therefore, aiming at the problems of the prior art and based on meeting the data requirements of deep learning mixed-precision training, the present invention provides a floating-point multiply-add unit with a fused precision conversion function and an application method thereof. By modifying the standard floating-point multiply-add operation unit to fuse the precision conversion function, it aims to reduce the hardware overhead and improve the implementation efficiency of deep learning algorithms, with the advantages of low overhead and high efficiency.

[0004] To solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0005] A floating-point multiply-add unit with a fused precision conversion function, comprising:

[0006] A data preprocessing module, which is used to extract the sign bit, exponent field, and mantissa field of the input floating-point operand according to the operation type, and complement the hidden bit of the unsigned mantissa in the mantissa field;

[0007] A mantissa multiplication module, which is used to multiply the unsigned mantissa with the hidden bit complemented to obtain a mantissa product;

[0008] An exponent difference module, which is used to subtract the exponent of the multiplication operand from the exponent of the addition operand to obtain an exponent difference value;

[0009] An addend alignment shift module, which is used to align and shift the addend according to the calculated exponent difference value;

[0010] An addition module, which is used to add the mantissa product and the addend after alignment and shifting to obtain a new mantissa;

[0011] A leading zero calculation module, which is used to calculate the number of leading zeros in the new mantissa;

[0012] A normalization shift module, which is used to perform a logical left shift on the new mantissa, and the amount of data shifted to the left is the number of leading zeros in the new mantissa calculated by the leading zero calculation module;

[0013] The floating-point multiply-add unit with a fused precision conversion function further includes:

[0014] An exponent adjustment module, which is used to perform exponent adjustment based on the number of leading zeros in the new mantissa, the difference between the exponent base value of the current mixed-precision operation addend precision and the target precision;

[0015] A mantissa rounding module, which is used to round the mantissa according to the target precision type to be supported;

[0016] A result output module, which is used to combine the sign bit, exponent, and rounded mantissa, and obtain a combined result for each precision, and select one of the combined results as the final output result according to the specific precision conversion control signal.

[0017] Optionally, the exponent adjustment module includes:

[0018] A difference calculation sub-module, which is used to calculate the exponent base value difference between the exponent base value of the current mixed-precision operation addend precision and the exponent base value of the target precision;

[0019] A merge addition sub-module, which is used to sum the number of leading zeros in the new mantissa and the exponent base value difference;

[0020] An exponent correction subtraction sub-module, which is used to subtract the sum calculated by the merge addition sub-module from the exponent difference value obtained by the exponent difference module to obtain a new exponent;

[0021] An overflow judgment sub-module, which is used to judge whether the new exponent overflows. If there is no overflow, the new exponent is directly output as the exponent after precision conversion. Otherwise, it jumps to execute the maximum / minimum value selection module.

[0022] The maximum / minimum value selection module is used when the new exponent overflows. If the new exponent is less than 0, 0 is selected as the exponent after precision conversion for output. If the new exponent is greater than the maximum value that the target precision exponent can represent, the maximum value that the target precision exponent can represent is selected as the exponent after precision conversion for output.

[0023] Optionally, when the overflow judgment sub-module judges whether the new exponent overflows, if the new exponent is less than 0 or greater than 31, it is determined that the new exponent overflows. Otherwise, it is determined that the new exponent does not overflow.

[0024] Optionally, the mantissa rounding module includes:

[0025] A control bit generation sub-module, which is used to generate corresponding rounding-related control bits for the mantissa after logical shifting by the normalized shifting module for multiple different target precisions. The rounding-related control bits include a glue bit G, a rounding bit R, and a least significant bit LSB;

[0026] A selection output sub-module, which is used to select a corresponding set of rounding-related control bits according to the target precision and output the corresponding rounded mantissa according to the rounding-related control bits;

[0027] A rounding carry judgment sub-module, which is used to generate a sign bit according to the rounded mantissa.

[0028] Optionally, the rounding carry judgment sub-module generating a sign bit according to the rounded mantissa includes: judging whether there is a carry in the rounded mantissa. When the rounded mantissa is all 1 and there is a carry in the rounding, the highest bit of the rounded mantissa needs to carry. If there is a carry in the highest bit of the rounded mantissa, the exponent is incremented by 1 and the rounded mantissa becomes all 0. The exponent is used as the sign bit, and the exponent, the rounded mantissa, and the sign bit are output to the result output module together.

[0029] Optionally, the multiple different target precisions include some or all of single precision, half precision, Bfloat16, and TensorFloat-32.

[0030] In addition, the present invention also provides a processor, which includes a processor body and a floating-point multiply-add unit provided in the processor body. The floating-point multiply-add unit is the floating-point multiply-add unit with the fusion precision conversion function described above.

[0031] In addition, the present invention also provides a computer device, including a processor and a memory connected to each other. The processor includes a processor body and a floating-point multiplier-accumulator disposed in the processor body, and the floating-point multiplier-accumulator is the floating-point multiplier-accumulator unit with a fused precision conversion function as described above.

[0032] In addition, the present invention also provides an application method for the floating-point multiplier-accumulator unit with a fused precision conversion function as described above, including:

[0033] S1. Extract a mixed-precision multiply-accumulate operation operator and a precision conversion operator based on the convolution or matrix multiplication operator of the target deep learning network model, where the mixed-precision multiply-accumulate operation operator is used to complete the mixed-precision multiply-accumulate operation, and the precision conversion operator is used to complete the precision conversion, including converting a low-precision floating-point number to a high-precision floating-point number and converting a high-precision floating-point number to a low-precision floating-point number;

[0034] S2. Fuse a pair of operators with the mixed-precision multiply-accumulate operation operator in the front and the precision conversion operator in the back into a fused operator. The fused operator is used to first complete the mixed-precision multiply-accumulate operation and then complete the precision conversion, and determine the target precision supported by the fused operator based on the application requirements of the target deep learning network model;

[0035] S3. Implement the fused operator using the floating-point multiplier-accumulator unit with a fused precision conversion function as described above to accelerate the convolution or matrix multiplication operation in the target deep learning network model.

[0036] Optionally, in step S1, the low-precision floating-point number is one of half-precision, Bfloat16, and TensorFloat-32, and the high-precision floating-point number is single-precision; when fusing into a fused operator in step S2, it includes fusing the mixed-precision multiply-accumulate operation operator in the forward propagation stage and the precision conversion operator for converting the high-precision floating-point number to a low-precision floating-point number after the forward propagation stage into a fused operator; and fusing the mixed-precision multiply-accumulate operation operator in the backward propagation stage and the precision conversion operator for converting the high-precision floating-point number to a low-precision floating-point number after the backward propagation stage into a fused operator.

[0037] Compared with the prior art, the present invention mainly has the following advantages:

[0038] In this embodiment, the precision conversion and floating-point multiply-accumulate are integrated into one arithmetic unit, which can effectively reduce the hardware overhead and improve the algorithm implementation efficiency. Specifically, it is reflected in the following aspects:

[0039] 1. Low hardware overhead: In the traditional implementation method, a combination of a floating-point multiply-add unit and a precision conversion unit is used, which requires designing two types of hardware circuits, resulting in a relatively large hardware overhead. In this embodiment, only a small amount of hardware logic needs to be added to the exponent adjustment and mantissa rounding parts of the floating-point multiply-add unit to achieve the same function, effectively reducing the hardware overhead.

[0040] 2. High execution efficiency: In the traditional method, through the multiply-add instruction, the floating-point multiply-add unit is called to calculate the multiply-add result, which is then written back to the storage component. Then, through the precision conversion instruction, the data is read from the memory and sent to the precision conversion unit, and after conversion, it is written back to the storage component again. During this process, three memory accesses are involved. In this embodiment, the precision conversion is directly performed during the floating-point multiply-add operation, and the converted result is directly written back to the storage component, only requiring one memory access, greatly improving the efficiency and simplifying the software control process. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a schematic structural diagram of a floating-point multiply-add unit with a fused precision conversion function according to the present invention.

[0042] Figure 2 It is a schematic structural diagram of an exponent adjustment module in an embodiment of the present invention.

[0043] Figure 3 It is a schematic structural diagram of a mantissa rounding module and a result output module in an embodiment of the present invention.

[0044] Figure 4 It is the training data stream of the target deep learning network model in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0045] As Figure 1 shown, the floating-point multiply-add unit with a fused precision conversion function in this embodiment has the same structural part as the existing floating-point multiply-add unit, including:

[0046] A data preprocessing module, which is used to extract the sign bit, exponent field, and mantissa field of the input floating-point operand according to the operation type, and supplement the hidden bit of the unsigned mantissa in the mantissa field;

[0047] A mantissa multiplication module, which is used to multiply the unsigned mantissa supplemented with the hidden bit to obtain the mantissa product;

[0048] An exponent difference module, which is used to subtract the exponent sum of the multiplication operand from the exponent of the addition operand to obtain the exponent difference;

[0049] An addend alignment shift module, which is used to align and shift the addend according to the calculated exponent difference;

[0050] An addition module, configured to add the mantissa product and the addend after alignment shifting to obtain a new mantissa;

[0051] A leading zero calculation module, configured to calculate the number of leading zeros in the new mantissa;

[0052] A normalization shifting module, configured to perform a logical left shift on the new mantissa, and the amount of data shifted left is the number of leading zeros in the new mantissa calculated by the leading zero calculation module;

[0053] Different from the structure of the existing floating-point multiply-add unit, the floating-point multiply-add unit with a fused precision conversion function in this embodiment further includes the following improved modules:

[0054] An exponent adjustment module, configured to perform exponent adjustment based on the number of leading zeros in the new mantissa, the difference between the current mixed-precision operation addend precision and the exponent base value of the target precision; in the traditional floating-point multiply-add unit, the exponent adjustment module only needs to consider the number of leading zeros, while in this embodiment with a combined precision conversion function, it also needs to consider the difference between the exponent base value of the high-precision result of the mixed-precision multiply-add and the exponent base value of the target precision;

[0055] A mantissa rounding module, configured to round the mantissa according to the type of target precision to be supported;

[0056] A result output module, configured to combine the sign bit, the exponent, and the rounded mantissa, and obtain a combined result for each precision, and select one of the combined results as the final output result according to the specific precision conversion control signal.

[0057] Based on the traditional floating-point multiply-accumulate unit, this embodiment adds a circuit for data precision conversion to integrate the two operations of floating-point multiply-accumulate and data precision conversion, meeting the requirements for the data stream format in deep learning mixed-precision training. The traditional floating-point multiply-accumulate operation is divided into multiple stages: including data preprocessing, mantissa multiplication, exponent difference calculation, addend alignment, addition, leading zero count, exponent adjustment, normalization shift, mantissa rounding, and result output. This embodiment will add support for precision conversion operations and intends to make improvements in the following three parts: exponent adjustment, mantissa rounding, and result output. The specific improvements are as follows: Exponent adjustment: In the traditional floating-point multiply-accumulate calculation process, the exponent needs to perform corresponding subtraction operations on the predicted exponent according to the result of the leading zero count to obtain the final result exponent. In the process of floating-point precision conversion, due to different exponent bases in different floating-point formats, the exponent also needs to perform addition / subtraction operations. For example, when converting single-precision floating-point to half-precision floating-point, the exponent base in the single-precision floating-point format is 127, while the exponent base in the half-precision floating-point format is 15. Therefore, the exponent needs to be subtracted by 112 (127 - 112) to obtain the converted exponent result. In this embodiment, to integrate the precision conversion function, the base difference of precision conversion is added to the result obtained from the leading zero count, and then the corresponding addition / subtraction operation is performed. In the process of converting high-precision data to low-precision, since the exponent bit field becomes smaller (for example, the exponent of a single-precision number is 8 bits, and the exponent of a half-precision number is 5 bits), an overflow situation may occur, and the handling of the overflow refers to the relevant regulations in the IEEE 754-2008 standard. Mantissa rounding: The way of mantissa rounding is similar to that of the traditional multiply-accumulate unit, using the roundTiesToEven rounding mode in the IEEE 754-2008 standard. In the traditional floating-point multiply-accumulate unit, only one rounding process is provided for one precision, and the rounding method is fixed. In this embodiment, rounding circuit logic needs to be designed for each possible target precision, and a mantissa result is obtained for each target precision. Result output: The adjusted exponent, the rounded mantissa result, and the sign bit are combined, and a combined result is obtained for each possible target output precision. According to the actual target precision type, one of them is selected as the output.

[0058] As Figure 2 shown, the exponent adjustment module includes:

[0059] A difference calculation sub-module for calculating the exponent base difference between the exponent base of the current mixed-precision operation addend precision and the exponent base of the target precision;

[0060] For example, if the current mixed-precision multiply-add mode is FP16×FP16+FP32, the multiply-add mode corresponds to the floating-point precision of matrices A, B, and C in the general matrix multiply-add operation of A×B+C respectively. The precision of the result obtained from the multiply-add calculation is FP32, and the exponent base value is 127. The target precision is FP16, the exponent base value is 15, and an additional subtraction of 112 (127 - 15) is required during exponent adjustment to obtain the converted exponent result;

[0061] The merge and add sub-module is used to sum the number of leading zeros in the new mantissa and the exponent base difference;

[0062] The exponent correction subtraction sub-module is used to subtract the sum calculated by the merge and add sub-module from the exponent difference obtained by the exponent difference module to obtain a new exponent;

[0063] The overflow judgment sub-module is used to judge whether the new exponent overflows. If it does not overflow, the new exponent is directly output as the exponent after precision conversion. Otherwise, it jumps to execute the maximum / minimum value selection module

[0064] The maximum / minimum value selection module is used when the new exponent overflows. If the new exponent is less than 0, 0 is selected as the exponent after precision conversion for output. If the new exponent is greater than the maximum value that can be represented by the target precision exponent, the maximum value that can be represented by the target precision exponent is selected as the exponent after precision conversion for output.

[0065] During the process of converting from high precision to low precision, the exponent may overflow. For example, when converting from single precision to half precision, the bit width of the exponent is reduced from 8 bits to 5 bits. If the adjusted exponent value is greater than 31 or less than 0, it indicates an overflow. In this embodiment, when the overflow judgment sub-module judges whether the new exponent overflows, if the new exponent is less than 0 or greater than 31, it is determined that the new exponent overflows. Otherwise, it is determined that the new exponent does not overflow.

[0066] Similar to the modification of the exponent adjustment part, Figure 3 This is the mantissa rounding method after integrating the precision conversion function in this embodiment, which can be configured to support the conversion of multiple data precisions. For example Figure 3 As shown, the mantissa rounding module includes:

[0067] The control bit generation sub-module (as shown in the upper dotted area in Figure 3 ) is used to generate corresponding rounding-related control bits for the mantissa after logical shifting by the normalized shift module for multiple different target precisions. The rounding-related control bits include the glue bit G, the rounding bit R, and the least significant bit LSB; as shown in Figure 3As shown, in this embodiment, multiple different target precisions include single-precision, half-precision, Bfloat16, and TensorFloat-32 (which can be some or more of them).

[0068] An output sub-module is selected to select a corresponding rounding-related control bit according to the target precision, and output a corresponding rounded mantissa according to the rounding-related control bit;

[0069] A rounding carry judgment sub-module (as shown in the lower dotted area in Figure 3 is used to generate a sign bit according to the rounded mantissa.

[0070] In this embodiment, the rounding carry judgment sub-module generates a sign bit according to the rounded mantissa, including: judging whether there is a carry in the rounded mantissa. When the rounded mantissa is all 1 and there is a carry in the rounding, the highest bit of the rounded mantissa needs to carry; if there is a carry in the highest bit of the rounded mantissa, then increment the exponent by 1 and the rounded mantissa becomes all 0; use the exponent as the sign bit, and output the exponent and the rounded mantissa to the result output module together.

[0071] In addition, this embodiment also provides a processor, including a processor body and a floating-point multiply-add unit provided in the processor body, and the floating-point multiply-add unit is the aforementioned floating-point multiply-add unit with a fused precision conversion function.

[0072] In addition, this embodiment also provides a computer device, including a processor and a memory connected to each other. The processor includes a processor body and a floating-point multiply-add unit provided in the processor body, and the floating-point multiply-add unit is the aforementioned floating-point multiply-add unit with a fused precision conversion function.

[0073] This embodiment designs a floating-point multiply-add unit with a fused precision conversion function, which is an improvement on the application of traditional floating-point multiply-add units in deep learning hardware platforms. The floating-point multiply-add unit with a fused precision conversion function in this embodiment is tightly coupled with deep learning algorithms. Therefore, in the implementation process, it is first necessary to analyze from the algorithm level, extract fused operators according to the target operator structure and network model data stream, and guide the design of customized arithmetic units with the functional requirements of the fused operators as the traction. Therefore, this embodiment also provides an application method for the aforementioned floating-point multiply-add unit with a fused precision conversion function, including:

[0074] S1. Extract a mixed-precision multiply-add operator and a precision conversion operator based on the convolution or matrix multiplication operator of the target deep learning network model. The mixed-precision multiply-add operator is used to perform mixed-precision multiply-add operations, and the precision conversion operator is used to perform precision conversion, including converting low-precision floating-point numbers to high-precision floating-point numbers and converting high-precision floating-point numbers to low-precision floating-point numbers. In this embodiment, the low-precision floating-point number in step S1 is one of half-precision, Bfloat16, and TensorFloat-32, and the high-precision floating-point number is single-precision.

[0075] S2. Fuse a pair of operators with the mixed-precision multiply-add operator in front and the precision conversion operator behind into a fused operator. The fused operator is used to first perform a mixed-precision multiply-add operation and then perform precision conversion, and determine the target precision that the fused operator needs to support based on the application requirements of the target deep learning network model.

[0076] S3. Implement the fused operator using the aforementioned floating-point multiply-add unit with a fused precision conversion function to accelerate the convolution or matrix multiplication operation in the target deep learning network model.

[0077] Figure 4 Describes the data flow between the operators in the network layers of the deep learning network model. Figure 4 In this, L represents the L-th network layer of the target deep learning network model, and the same applies to L + 1 and L - 1.

[0078] When fusing into a fused operator in step S2 of this embodiment, it includes fusing the mixed-precision multiply-add operator in the forward propagation stage and the precision conversion operator for converting high-precision floating-point numbers to low-precision floating-point numbers after the forward propagation stage into a fused operator; and fusing the mixed-precision multiply-add operator in the backward propagation stage and the precision conversion operator for converting high-precision floating-point numbers to low-precision floating-point numbers after the backward propagation stage into a fused operator. Figure 4 The operators involved mainly include: the mixed-precision multiply-add operator 10 in the three stages of forward propagation, backward propagation, and weight (gradient) update, the precision conversion operator 11 in the forward propagation process, the precision conversion operator 12 in the backward propagation process, and the weight precision conversion operator 13 in the forward propagation process.

[0079] The mixed-precision multiply-accumulate operator 10 is the core operator in the three stages (forward propagation, backward propagation, and weight (gradient) update) during the training process of deep learning algorithms. It mainly corresponds to the convolution or matrix multiplication operator. Such an operator is the most core operator in deep learning algorithms. Its multiplication operands use a low-precision floating-point format, the addition operands use a high-precision floating-point format, and the output is a high-precision floating-point multiply-accumulate result. The low precision in deep learning algorithms mainly includes: half-precision, Bfloat16, and TensorFloat-32, and the high precision often refers to single precision. The data format is determined according to the specific requirements of the algorithm model.

[0080] The purpose of the precision conversion operator 11 during forward propagation is to convert the high-precision multiply-accumulate operation result of the current network layer into a low-precision format. By writing back the low-precision data to the storage, the storage overhead can be effectively reduced, and it can also be directly used as the activation value and input to the next network layer. The operation data type of the precision conversion here corresponds to the data format selected in the mixed-precision multiply-accumulate operator 10.

[0081] The precision conversion operator 12 during backward propagation is used for the precision conversion operation of the multiply-accumulate result during backward propagation. Its principle is the same as that of the precision conversion operator 11 during forward propagation.

[0082] The weight precision conversion operator 13 during forward propagation is used for the precision conversion operation of the weight before floating-point multiply-accumulate during forward propagation. The format of the weight in the storage is high-precision (single precision). When performing mixed-precision multiply-accumulate operations, it needs to be converted to low precision first and then perform the operation. The precision conversion here is not fused with the floating-point multiply-accumulate because this is the preprocessing of the operation data. The precision conversion precedes the floating-point multiply-accumulate, and the hardware logic in the floating-point multiply-accumulate cannot be reused on the data stream. Fusing the two will not save hardware overhead.

[0083] The calculation order among the above-mentioned operators determines whether the common calculation parts among them can be fused, that is, whether a fused operator can be formed. Figure 4 The mixed-precision multiply-accumulate operator 10 and the precision conversion operator 11 during forward propagation, or the mixed-precision multiply-accumulate operator 10 and the precision conversion operator 12 during backward propagation can be fused because they meet the requirement that the floating-point multiply-accumulate is before and the precision conversion is after on the data stream, and the exponent adjustment and mantissa rounding calculations required by both can be fused. However, changing their order will make it impossible to fuse. For example, the mixed-precision multiply-accumulate operator 10 and the weight precision conversion operator 13 during forward propagation cannot form a fused operator. According to Figure 4Extract the fused operator from the data stream of the deep learning network model, and at the same time determine the data precision supported by the fused operator according to the specific application requirements. Modify based on the traditional floating-point multiply-add structure, and design a new arithmetic unit to realize the fusion of floating-point multiply-add and precision conversion.

[0084] Generally speaking, this embodiment specifically implements a floating-point arithmetic unit. Compared with the traditional floating-point multiply-add unit, there is only one more enable control signal for precision conversion at the external interface level. When this embodiment is applied to the deep learning hardware platform, directly replace the original independent floating-point multiply-add unit and precision conversion unit, and at the same time merge and connect the interfaces of these two arithmetic units to the arithmetic unit of this embodiment. The replacement process is very simple. In the actual application process, this part of the replacement is transparent to the programmer and will not increase the control difficulty of the programmer. Multiply-accumulate operation and precision conversion are one of the most common operations in deep learning. The traditional implementation method classifies these two types of operations into different operators and designs dedicated arithmetic units respectively, with large hardware overhead and low operator implementation efficiency. This embodiment combines the floating-point multiply-add and precision conversion operations into one, and only needs to make a small modification to the traditional floating-point multiply-add unit and increase a small amount of hardware overhead, which can effectively improve the computing efficiency. Coupling the two types of operators together is consistent with the deep learning algorithm data, which can simplify the algorithm implementation and reduce the control difficulty.

[0085] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code. The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate for implementing in the process Figure 1 One process or multiple processes and / or blocks Figure 1Apparatus for the functions specified in one or more boxes. These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including an instruction apparatus that implements the functions specified in the process Figure 1 One process or more processes and / or boxes Figure 1 Apparatus for the functions specified in one or more boxes. These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the process Figure 1 One process or more processes and / or boxes Figure 1 Apparatus for the functions specified in one or more boxes.

[0086] The above is only the preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the idea of the present invention belong to the protection scope of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A floating-point multiply-add unit with a fused precision conversion function, comprising: A data preprocessing module, configured to extract the sign bit, exponent field, and mantissa field of the input floating-point operand according to the operation type, and supplement the hidden bit of the unsigned mantissa in the mantissa field; A mantissa multiplication module, configured to multiply the unsigned mantissa supplemented with the hidden bit to obtain a mantissa product; An exponent difference module, configured to subtract the exponent sum of the multiplication operand from the exponent of the addition operand to obtain an exponent difference value; An addend alignment shift module, configured to perform alignment shift on the addend according to the calculated exponent difference value; An addition module, configured to add the mantissa product and the addend after alignment shift to obtain a new mantissa; A leading zero calculation module, configured to calculate the number of leading zeros in the new mantissa; A normalization shift module, configured to perform a logical left shift on the new mantissa, and the amount of data shifted left is the number of leading zeros in the new mantissa calculated by the leading zero calculation module; Characterized in that it further comprises: An exponent adjustment module, configured to perform exponent adjustment based on the number of leading zeros in the new mantissa, the difference between the exponent base value of the current mixed-precision operation addend precision and the target precision; A mantissa rounding module, configured to round the mantissa according to the target precision type to be supported; A result output module, configured to combine the sign bit, exponent, and rounded mantissa, and obtain a combined result for each precision, and select one of the combined results as the final output result according to the specific precision conversion control signal; The exponent adjustment module includes: A difference calculation sub-module, configured to calculate the exponent base value difference between the exponent base value of the current mixed-precision operation addend precision and the exponent base value of the target precision; A merge addition sub-module, configured to sum the number of leading zeros in the new mantissa and the exponent base value difference; An exponent correction subtraction sub-module, configured to subtract the sum calculated by the merge addition sub-module from the exponent difference value obtained by the exponent difference module to obtain a new exponent; An overflow judgment sub-module, configured to judge whether the new exponent overflows. If it does not overflow, directly output the new exponent as the exponent after precision conversion. Otherwise, jump to execute the maximum / minimum value selection module; A maximum / minimum value selection module, configured to, when the new exponent overflows, if the new exponent is less than 0, select 0 as the exponent after precision conversion and output it. If the new exponent is greater than the maximum value that the target precision exponent can represent, select the maximum value that the target precision exponent can represent as the exponent after precision conversion and output it.

2. The floating-point multiply-add unit with a fused precision conversion function according to claim 1, characterized in that, When the overflow judgment sub-module judges whether the new exponent overflows, if the new exponent is less than 0 or greater than 31, it is determined that the new exponent overflows. Otherwise, it is determined that the new exponent does not overflow.

3. The floating-point multiply-add unit with a fusion precision conversion function according to claim 1, characterized in that, The mantissa rounding module includes: A control bit generation sub-module, configured to generate corresponding rounding-related control bits for the mantissa after logical shift by the normalization shift module for multiple different target precisions. The rounding-related control bits include a glue bit G, a rounding bit R, and a least significant bit LSB; A selection output sub-module, configured to select a corresponding set of rounding-related control bits according to the target precision, and output the corresponding rounded mantissa according to the rounding-related control bits; The rounding carry judgment sub-module is used to generate a sign bit according to the rounded mantissa.

4. The floating-point multiply-add unit with a fusion precision conversion function according to claim 3, characterized in that, The rounding carry judgment sub-module generating a sign bit according to the rounded mantissa includes: judging whether there is a carry in the rounded mantissa. When the rounded mantissa is all 1s and there is a carry in the rounding, the highest bit of the rounded mantissa needs to carry; if there is a carry in the highest bit of the rounded mantissa, then increment the exponent by 1 and the rounded mantissa becomes all 0s; use the exponent as the sign bit and output it together with the exponent and the rounded mantissa to the result output module.

5. The floating-point multiply-add unit with a fusion precision conversion function according to claim 3, characterized in that, The various different target precisions include some or all of single precision, half precision, Bfloat16, and TensorFloat-32.

6. A processor, comprising a processor body and a floating-point multiply-accumulator disposed in the processor body, characterized in that, The floating-point multiply-accumulator is the floating-point multiply-accumulate unit with a fused precision conversion function described in any one of claims 1 to 5.

7. A computer device, comprising a processor and a memory connected to each other, the processor including a processor body and a floating-point multiply-accumulator provided in the processor body, characterized in that, The floating-point multiply-accumulator is the floating-point multiply-accumulate unit with a fused precision conversion function described in any one of claims 1 to 5.

8. A method for applying a floating-point multiply-add unit with a fusion precision conversion function according to any one of claims 1 to 5, characterized in that, It includes: S1, extracting a mixed-precision multiply-accumulate operation operator and a precision conversion operator based on the convolution or matrix multiplication operator of the target deep learning network model, where the mixed-precision multiply-accumulate operation operator is used to complete the mixed-precision multiply-accumulate operation, and the precision conversion operator is used to complete the precision conversion, including converting a low-precision floating-point number to a high-precision floating-point number and converting a high-precision floating-point number to a low-precision floating-point number; S2, fusing a pair of operators with the mixed-precision multiply-accumulate operation operator in front and the precision conversion operator behind into a fused operator. The fused operator is used to first complete the mixed-precision multiply-accumulate operation and then complete the precision conversion, and determine the target precision that the fused operator needs to support based on the application requirements of the target deep learning network model; S3, implementing the fused operator using the floating-point multiply-accumulate unit with a fused precision conversion function described in any one of claims 1 to 5 to accelerate the convolution or matrix multiplication operation in the target deep learning network model.

9. The application method of the floating-point multiply-add unit with a fusion precision conversion function according to claim 8, characterized in that, In step S1, the low-precision floating-point number is one of half precision, Bfloat16, and TensorFloat-32, and the high-precision floating-point number is single precision; when fusing into a fused operator in step S2, it includes fusing the mixed-precision multiply-accumulate operation operator in the forward propagation stage and the precision conversion operator for converting the high-precision floating-point number to a low-precision floating-point number after the forward propagation stage into a fused operator; and fusing the mixed-precision multiply-accumulate operation operator in the backward propagation stage and the precision conversion operator for converting the high-precision floating-point number to a low-precision floating-point number after the backward propagation stage into a fused operator.

Images