Multiplying and adding operation hardware with mixed precision

Abstract

The invention discloses mixed precision multiply-add operation hardware, which comprises an input processing unit used for obtaining standard symbols, standard indexes and standard mantissas according to a pre-configured input format and unpacked input multiply-add operation factors, and obtaining input and output special value detection results; the floating point multiplier-adder is used for calculating an operation result after the operand multiplies the scaling factor and the offset is accumulated according to the symbol, the exponent and the mantissa of each multiplication-addition operation factor; and the output processing unit is used for obtaining a target format result according to a pre-configured output format, a rounding type and a normalization and rounding operation result, packaging the target format result according to a detection result and outputting the target format result, and by multiplexing a data path of the floating-point multiplier-adder, the chip area is saved, format conversion is integrated, a data stream is simplified, and the data processing efficiency is improved. The requirements for automatic reconstruction of the multiply-add operation unit according to input and output formats and quantization and inverse quantization of floating points and integers are met, and the reuse rate of computing resources is increased.

Description

Technical Field

[0001] The present invention relates to the technical field of integrated chips, and in particular to mixed-precision multiplication and addition hardware. Background Art

[0002] Multiplication and addition is one of the most basic operations in deep learning. The current mainstream deep learning algorithms generally support mixed-precision training and inference modes. In order to improve hardware throughput and computing power, low-precision data formats are used for input and output, and high-precision data formats are used for internal accumulation.

[0003] Usually in floating-point mode, the input, scaling factor, offset and output of the multiplication and addition operation are all floating-point format data, and in integer mode, the input and output of the multiplication and addition operation are integer format data, and the scaling factor and offset are all floating-point format data. For the multiplication and addition operations in floating-point mode and integer mode, the existing technical solutions are usually implemented using independent data paths. In floating-point mode, the floating-point data path is used for calculation, and in integer mode, the integer data path is used for calculation. That is, when one mode is used, the computing resources of the other mode are idle, which increases the chip area requirement and reduces the reuse rate of computing resources. Summary of the invention

[0004] The present invention provides a mixed-precision multiplication-addition hardware to solve the problems of large chip area requirement and low reuse rate of computing resources caused by using independent data paths in the mixed-precision multiplication-addition process.

[0005] According to one aspect of an embodiment of the present invention, there is provided a mixed-precision multiplication-addition hardware, comprising: an input processing unit, a floating-point multiplier-adder, and an output processing unit, wherein:

[0006] An input processing unit, used to unpack each input multiplication and addition operation factor according to a pre-configured input format, obtain a standard sign, a standard exponent and a standard mantissa corresponding to each multiplication and addition operation factor, and obtain an input and output special value detection result;

[0007] A floating-point multiplier-adder, for calculating the floating-point multiplication-addition result obtained by multiplying the operand by the scaling factor and accumulating the offset according to the standard sign, standard exponent and standard mantissa of each multiplication-addition operation factor;

[0008] An output processing unit, used to normalize and round the floating-point multiplication and addition operation results according to a pre-configured output format and rounding type to obtain a target format result, and to package and output the target format result according to an input and output special value detection result;

[0009] Among them, the types of multiplication and addition operation factors include: operands, scaling factors and offsets, the types of input formats and output formats include floating-point formats or integer formats with set bit widths of each multiplication and addition operation factor, and the bit width of the floating-point format includes: exponent bit width and fraction bit width.

[0010] The mixed-precision multiplication and addition hardware of the embodiment of the present invention includes: an input processing unit, a floating-point multiplier and adder, and an output processing unit, wherein: the input processing unit is used to unpack each input multiplication and addition operation factor according to a pre-configured input format, obtain the standard sign, standard exponent and standard mantissa corresponding to each multiplication and addition operation factor, and obtain the input and output special value detection result; the floating-point multiplier and adder is used to calculate the floating-point multiplication and addition operation result obtained by multiplying the operand by the scaling factor and accumulating the offset according to the standard sign, standard exponent and standard mantissa of each multiplication and addition operation factor; the output processing unit is used to normalize and round the floating-point multiplication and addition operation result according to the pre-configured output format and rounding type, obtain the target format result, and pack and output the target format result according to the input and output special value detection result. By reusing the data path of the floating-point multiplier and adder, the chip area is saved, the format conversion is integrated, the data flow is simplified, the automatic reconstruction of the multiplication and addition operation unit according to the input and output format and the requirements for quantization and dequantization of floating points and integers are realized, and the reuse rate of computing resources is improved.

[0011] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 is a schematic diagram of the structure of mixed-precision multiplication-addition hardware provided according to Embodiment 1 of the present invention;

[0014] Figure 2 is a schematic diagram of the structure of another mixed-precision multiplication-addition hardware provided according to the second embodiment of the present invention;

[0015] Figure 3 is a schematic diagram of the structure of another mixed-precision multiplication-addition hardware provided according to Embodiment 3 of the present invention;

[0016] Figure 4 It is a flowchart of a multi-precision multiplication and addition operation applicable to the embodiment of the present invention;

[0017] Figure 5 is a flow chart of a mixed-precision multiplication-addition operation applicable to an embodiment of the present invention;

[0018] Figure 6 is a schematic diagram of a preferred input data arrangement scheme applicable to an embodiment of the present invention;

[0019] Figure 7 is a schematic diagram of a preferred output result arrangement scheme applicable to an embodiment of the present invention;

[0020] Figure 8 is a schematic diagram of a floating point data format applicable to an embodiment of the present invention;

[0021] Fig. 9 is a schematic diagram of an unpacking result applicable to an embodiment of the present invention;

[0022] Fig.10 is a schematic diagram of a mantissa applicable to an embodiment of the present invention;

[0023] Fig.11 is a schematic diagram of another mantissa applicable to the embodiment of the present invention;

[0024] Fig.12 is a schematic diagram of a bit width during summation applicable to an embodiment of the present invention;

[0025] Fig.13 is a schematic diagram of a bit arrangement of a product applicable to an embodiment of the present invention;

[0026] Fig.14 Schematic diagram of a multiplication-addition result with a high-bit padded sign bit applicable to an embodiment of the present invention;

[0027] Fig.15 Schematic diagram of the decimal point position of a floating-point multiplication and addition result applicable to an embodiment of the present invention;

[0028] Fig.16 is a schematic diagram of a normalized decimal point position applicable to an embodiment of the present invention;

[0029] Fig.17 is a schematic diagram of another normalized decimal point position applicable to an embodiment of the present invention;

[0030] Fig.18 is a schematic diagram of the bit arrangement of normalized multiplication and addition results in a rounding module applicable to an embodiment of the present invention;

[0031] Fig.19is a schematic diagram of a rounded multiplication and addition result applicable to an embodiment of the present invention;

[0032] Fig. 20 It is a schematic diagram of an output result applicable to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] Embodiment 1

[0036] Figure 1 This is a schematic diagram of the structure of a mixed-precision multiplication and addition hardware provided in the first embodiment of the present invention. This embodiment can be applied to meet the needs of quantizing and dequantizing floating points and integers, especially in deep learning scenarios. Figure 1 As shown, the mixed-precision multiplication and addition hardware includes: an input processing unit 110, a floating-point multiplier and adder 120, and an output processing unit 130, wherein:

[0037] The input processing unit 110 is used to unpack each input multiplication and addition operation factor according to a pre-configured input format, obtain a standard sign, a standard exponent and a standard mantissa corresponding to each multiplication and addition operation factor, and obtain an input and output special value detection result;

[0038] The floating-point multiplier-adder 120 is used to calculate the floating-point multiplication-addition result obtained by multiplying the operand by the scaling factor and accumulating the offset according to the standard sign, standard exponent and standard mantissa of each multiplication-addition operation factor;

[0039] The output processing unit 130 is used to normalize and round the floating-point multiplication and addition operation results according to the pre-configured output format and rounding type to obtain the target format result, and to package and output the target format result according to the input and output special value detection result;

[0040] Among them, the types of multiplication and addition operation factors include: operands, scaling factors and offsets, the types of input formats and output formats include floating-point formats or integer formats with set bit widths of each multiplication and addition operation factor, and the bit width of the floating-point format includes: exponent bit width and fraction bit width.

[0041] In the embodiment of the present invention, the multiplication and addition operation factor refers to the input operand of the multiplication and addition operation, including: operand, scaling factor and offset. Among them, the operand refers to the number to be multiplied in the multiplication and addition operation, which can be an integer or a floating point number. The scaling factor is a factor used to adjust the magnitude or precision of the multiplication and addition operation result and multiply the operand. The offset is used to adjust the range of the multiplication and addition operation result or perform normalization processing, and is added to the multiplication and addition operation result. The output result of the multiplication and addition operation can be operand × scaling factor + offset.

[0042] The input format can be specifically understood as the data format of the input operand, which can be a floating-point data format or an integer data format, including a floating-point format or an integer format with a set bit width for each multiplication and addition operation factor. The bit width determines the range and precision of the value. In the integer format, the bit width is used to determine the number of bits of the integer, including the sign bit and the value bit. In the floating-point format, the bit width includes: the exponent bit width and the fraction bit width. Among them, the exponent bit width determines the range of values ​​that the floating-point number can represent. The larger the exponent bit width, the wider the range of values ​​that can be represented. For example, the exponent bit width of a single-precision floating-point number (32 bits) is 8 bits, and the exponent bit width of a double-precision floating-point number (64 bits) is 11 bits. The fraction bit width, also known as the mantissa bit width, determines the precision of the floating-point number. The larger the fraction bit width, the more accurate the value that can be represented. The fraction bit width of a single-precision floating-point number is 23 bits, and the fraction bit width of a double-precision floating-point number is 52 bits.

[0043] Generally speaking, the input operand supports floating point data format and integer data format, and its bit width can be the maximum value of the bit width of the floating point data format and the integer data format. For example, if the floating point (FP) data format is FP16 (16-bit floating point format) and the integer (INT) data format is INT32 (32-bit integer format), the bit width of the input operand is 32 bits.

[0044] Unpacking refers to the unpacking of scaling factors and offsets, as well as the parsing and processing of operands. In floating-point mode, the input operands, scaling factors, offsets, and output results of the multiplication and addition operations are in floating-point format data. In integer mode, the input operands and output results of the multiplication and addition operations are in integer format data, and the scaling factors and offsets are both in floating-point format data.

[0045] The standard sign, standard exponent and standard mantissa may be specifically understood as: the corresponding sign, exponent and mantissa in a unified format obtained by unpacking the scaling factor, offset and input operand by the input processing unit 110 .

[0046] Generally speaking, the input processing unit 110 unpacks the scaling factor, offset and input operand to obtain the corresponding sign, exponent and mantissa. Among them, the sign part can directly output the highest bit of the input parameter. The exponent bit is usually represented by an offset binary encoding method, and the actual exponent value requires the binary exponent bit to be converted to decimal and subtracted from the offset to obtain. For non-zero exponent values, the mantissa bits usually do not include the front implicit 1, and will be added to the front of the mantissa during decoding. The actual mantissa value needs to be converted from the binary value to decimal and multiplied by a negative power of 2 to obtain, where the power of each bit decreases from the highest bit to the lowest bit, thereby obtaining a unified format of sign, exponent and mantissa, that is, a standard sign, a standard exponent and a standard mantissa.

[0047] The input and output special value detection results can be specifically understood as: the input processing unit 110 detects whether the input operand contains the three special values ​​of 0, infinity and non-number, and detects whether the output result is infinity and non-number (used to indicate that it is not a valid number). For example, the input and output values ​​to be detected can be converted into decimal for comparison and judgment. By detecting the input and output special values, these special values ​​can be prevented from affecting the logic execution of the subsequent program and the accuracy of the output results.

[0048] The rounding type can be specifically understood as: it refers to the rounding method used when rounding the exact calculation result (equivalent to infinite bit width) to the output result of finite bit width during the numerical calculation process, which can specifically include: rounding to 0, rounding to positive infinity, rounding to negative infinity, and rounding to the nearest even number. Among them, rounding to 0 means rounding the value to the nearest integer, but always rounding to zero, that is, positive numbers become smaller after rounding, and negative numbers become larger after rounding. For example, 3.7 is rounded to 0 to 3, and -3.7 is rounded to 0 to -3. Rounding to positive infinity means rounding the value to the nearest integer, but always rounding to positive infinity, that is, both positive and negative numbers will be rounded to a larger value. For example, 3.2 is rounded to positive infinity to 4, and -3.2 is rounded to positive infinity to -3. Rounding to negative infinity means rounding the value to the nearest integer, but always rounding to negative infinity. That is, positive numbers are rounded down and negative numbers are rounded up. For example, 3.8 rounded to -3 is 3, and -3.8 rounded to -4. Rounding to the nearest even means rounding the value to the nearest integer. If the value is exactly halfway between two integers, it will be rounded to the nearest even integer. For example, 2.5 rounded to the nearest even is 2, and 3.5 rounded to the nearest even is 4. Because the rounding direction is not biased in one direction, this type of rounding helps reduce the accumulation of rounding errors.

[0049] The output format can be specifically understood as: the data format of the output result, including multiple floating point data formats and multiple integer data formats. The bit width of the output result can be determined by the maximum bit width of all supported output formats. For example, the data format of the output result supports FP32, FP16, BF16 (Brain Floating Point, here refers to 16-bit brain floating point), INT32 and INT16 data formats, and the bit width of the output result is 32 bits. Normalization can be specifically understood as: unifying the results of multiplication and addition operations into a standard form.

[0050] Generally speaking, the output processing unit 130 first detects the number of redundant sign bits of the floating-point multiplication and addition operation result, determines the number of sign bit padding bits according to the maximum bit width in the integer format supported by the output and the maximum fractional bit width in the floating-point format supported by the output, and pads the high-order sign bits of the floating-point multiplication and addition result with the number of padding bits while detecting the redundant sign bits, so as to avoid the problem that the shift in two directions, left shift and right shift, may be required when the output format is an integer. According to the output format, the position of the decimal point after normalization is determined. In the representation of floating-point numbers, the position of the decimal point is determined relative to the mantissa and the exponent, and the decimal point is shifted to the left for normalization. For example, if the highest bit of the mantissa is not 1, the decimal point needs to be moved to the left until it is located after the highest bit 1. Then, the exponent value is modified according to the number of bits shifted to the left, because shifting the decimal point to the left is actually dividing the value by 2, and the exponent represents the power of 2, so the exponent value is reduced by 1 for each bit shifted to the left. Therefore, if the shift is n bits to the left, the exponent value needs to be subtracted by n. Determine the carry value of the normalized mantissa according to the rounding mode, and calculate the rounded mantissa. Generate the final required result according to the output format, the rounded multiplication and addition result, the corrected exponent, and the special result detection value.

[0051] Specifically, after inputting the multiplication and addition operation factors, the input processing unit 110 performs unpacking processing on each input multiplication and addition operation factor according to the pre-configured input format, obtains the standard sign, standard exponent and standard mantissa corresponding to each multiplication and addition operation factor, and obtains the input and output special value detection result, which is used for the output processing unit 130 to determine whether the input special result detection value is valid to determine the output result. According to the number of bits set by the pre-configured input format, the processing result of the input processing unit 110 is output to realize the conversion of the input operands in floating point or integer format into a unified intermediate format, which is convenient for subsequent calculations. Then, the floating-point multiplier 120 calculates the floating-point multiplication and addition operation result obtained by multiplying the operand by the scaling factor according to the standard sign, standard exponent and standard mantissa of each multiplication and addition operation factor, and accumulates the offset, which includes the exponent part and the mantissa part. Finally, the output processing unit 130 performs normalization and rounding processing on the floating-point multiplication and addition operation result according to the pre-configured output format and rounding type, obtains the target format result, and packages and outputs the target format result according to the input and output special value detection result.

[0052] The mixed-precision multiplication and addition hardware of the embodiment of the present invention includes: an input processing unit, a floating-point multiplier and adder, and an output processing unit, wherein: the input processing unit is used to unpack each input multiplication and addition operation factor according to a pre-configured input format, obtain the standard sign, standard exponent and standard mantissa corresponding to each multiplication and addition operation factor, and obtain the input and output special value detection result; the floating-point multiplier and adder is used to calculate the floating-point multiplication and addition operation result obtained by multiplying the operand by the scaling factor and accumulating the offset according to the standard sign, standard exponent and standard mantissa of each multiplication and addition operation factor; the output processing unit is used to normalize and round the floating-point multiplication and addition operation result according to the pre-configured output format and rounding type, obtain the target format result, and pack and output the target format result according to the input and output special value detection result. By reusing the data path of the floating-point multiplier and adder, the chip area is saved, the format conversion is integrated, the data flow is simplified, the automatic reconstruction of the multiplication and addition operation unit according to the input and output format and the requirements for quantization and dequantization of floating points and integers are realized, and the reuse rate of computing resources is improved.

[0053] Optionally, based on the above embodiments, the input processing unit 110 includes: a mixed precision unpacking subunit and a special value detection subunit;

[0054] The mixed precision unpacking subunit is used to unpack each input multiplication and addition operation factor according to the type of the multiplication and addition operation factor and the input format, and obtain the standard sign, standard exponent and standard mantissa corresponding to each multiplication and addition operation factor respectively;

[0055] The special value detection subunit is used to perform special value detection on each input multiplication and addition operation factor, and perform special value detection on the output result according to the input special value detection result to obtain the output special value detection result.

[0056] Specifically, the mixed precision unpacking subunit unpacks the input operands, scaling factors, and offsets, and the unpacked content includes the sign, exponent, and fraction. When unpacking, the sign part takes the highest bit of the input, and no special processing is required. The sign can be directly output, that is, the standard sign; the exponent part needs to be processed according to the value of the fraction part. For example: when the exponent is 0 and the fraction is not 0, 1 is used as the standard exponent, otherwise, the exponent part is directly used as the standard exponent; when the exponent part is 0, the integer 0 is added before the highest bit of the fraction, and the result of the 0-filled result is used as the standard mantissa; otherwise, the integer 1 is added before the highest bit of the fraction, and the result of the 1-filled result is used as the standard mantissa.

[0057] The special value detection subunit performs special value detection on input operands, scaling factors and offsets, such as 0, infinity and non-number (NaN), and detects special multiplication and addition results, such as the result of multiplying any number with zero is zero; the result of multiplying infinity with zero is NaN; the result of adding zero to any number is the number itself; when a non-zero number is multiplied by infinity, the result is positive infinity or negative infinity, depending on the sign of the multiplier; adding positive infinity or negative infinity to any positive or negative number results in positive infinity or negative infinity, respectively; the result of adding positive infinity to negative infinity is NaN; the result of multiplying or adding any number to NaN is NaN, where NaN is used to represent undefined or unrepresentable values, such as 0 / 0, ∞-∞, etc.

[0058] Based on the above embodiments, the mixed precision unpacking subunit is specifically used for:

[0059] If the type of the first multiplication-addition operation factor currently being unpacked is a scaling factor or an offset, parsing out a first sign, a first exponent, and a first fraction matching the first multiplication-addition operation factor;

[0060] Taking the first sign as the standard sign of the factor of the first multiplication-addition operation;

[0061] When the first exponent is 0 and the first fraction is not 0, 1 is used as the standard exponent of the first multiplication-addition operation factor; otherwise, the first exponent is used as the standard exponent of the first multiplication-addition operation factor;

[0062] When the first exponent is 0, the integer 0 is padded before the highest digit of the first fraction, and the result of the padded 0 is used as the standard mantissa of the first multiplication and addition operation factor; otherwise, the integer 1 is padded before the highest digit of the first fraction, and the result of the padded 1 is used as the standard mantissa of the first multiplication and addition operation factor.

[0063] In the embodiment of the present invention, the first multiplication-addition operation factor can be specifically understood as: the multiplication-addition operation factor currently being processed, where the first is only used for distinction and does not represent the order of precedence. The first sign, the first exponent and the first score can be specifically understood as: the sign, exponent and score that match the first multiplication-addition operation factor, that is, the sign, exponent and score that match the scaling factor or offset.

[0064] Specifically, if the type of the first multiplication-addition operation factor currently being unpacked is a scaling factor or an offset, the unpacking method for the scaling factor and the offset may be: parse out the first sign, the first exponent and the first fraction that match the first multiplication-addition operation factor, the sign part does not need special processing and can be directly output, that is, the first sign is used as the standard sign of the first multiplication-addition operation factor; the exponent part needs to be processed according to the value of the fraction part, specifically: when the exponent value is 0 and the fraction value is not 0, the exponent is modified to 1, otherwise, the exponent value is not modified, that is, when the first exponent is 0 and the first fraction is not 0, 1 is used as the standard exponent of the first multiplication-addition operation factor, otherwise, the first exponent is used as the standard exponent of the first multiplication-addition operation factor; the mantissa part is formed together according to the values ​​of the exponent part and the fraction part, specifically: if the exponent part is 0, it is necessary to fill the highest bit of the fraction part with an integer 0, otherwise fill the integer 1, that is, when the first exponent is 0, after filling the highest bit of the first fraction with an integer 0, the result of filling 0 is used as the standard mantissa of the first multiplication-addition operation factor, otherwise, after filling the highest bit of the first fraction with an integer 1, the result of filling 1 is used as the standard mantissa of the first multiplication-addition operation factor.

[0065] Further, based on the above embodiments, the mixed precision unpacking subunit may be specifically used for:

[0066] If the type of the second multiplication-addition operation factor currently being unpacked is an operand, when the input format of the operand is an integer format, a second sign and an integer mantissa matching the second multiplication-addition operation factor are parsed; when the input format of the operand is a floating-point format, a second sign, a second exponent, and a second fraction matching the second multiplication-addition operation factor are parsed;

[0067] The second sign is used as the standard sign of the factor of the second multiplication-addition operation;

[0068] If the input format for the operand is a floating point format, when the second exponent is 0 and the second fraction is not 0, 1 is used as the standard exponent of the second multiplication-addition operation factor; when the second exponent is not 0 or the second fraction is 0, the second exponent is used as the standard exponent of the second multiplication-addition operation factor;

[0069] If the input format for the operand is an integer format, when the integer mantissa is 0, 0 is used as the standard exponent of the second multiplication and addition operation factor; when the integer mantissa is not 0, Standard exponent as the factor of the second multiply-add operation;

[0070] Among them, WA I is the bit width in the operand integer format, WEA F The exponent width of the operand floating point format;

[0071] in WA IGreater than or equal to (WEA F +2), if the input format for the operand is the floating-point format, then when the second exponent is 0, 00 is added to the front of the highest bit of the second fraction, and the target number of 0s is added to the lowest bit of the second fraction, and the supplemented result is used as the standard mantissa of the second multiplication and addition operation factor; when the second exponent is 1, 01 is added to the front of the highest bit of the second fraction, and the target number of 0s is added to the lowest bit of the second fraction, and the supplemented result is used as the standard mantissa of the second multiplication and addition operation factor; if the input format for the operand is the integer format, the combination of the second sign and the integer mantissa is used as the standard mantissa of the second multiplication and addition operation factor;

[0072] in WA I Less than (WEA F +2), if the input format for the operand is a floating-point format, then when the second exponent is 0, 00 is added to the top bit of the second fraction, and the added result is used as the standard mantissa of the second multiplication-addition operation factor; when the second exponent is 1, 01 is added to the top bit of the second fraction, and the added result is used as the standard mantissa of the second multiplication-addition operation factor; if the input format for the operand is an integer format, the second sign and the integer mantissa are combined, and the lowest bit of the combined result is filled with 0 of the target number of bits, and the added result is used as the standard mantissa of the second multiplication-addition operation factor;

[0073] Among them, the target number of digits is |(WEA F +2)-WAI|, |.| is the symbol for taking absolute value.

[0074] In the embodiment of the present invention, the second multiplication-addition operation factor can be specifically understood as: the multiplication-addition operation factor currently being processed, where the second is only used for distinction and does not represent the order of precedence. The second sign, the second exponent and the second fraction can be specifically understood as: the sign, exponent and fraction that match the second multiplication-addition operation factor, that is, the sign, exponent and fraction that match the operand in floating-point format. Among them, the second sign can also refer to the sign that matches the operand in integer format.

[0075] Specifically, if the type of the second multiplication-addition operation factor currently being unpacked is an operand, the unpacking method for the input operand is related to the input format. When the input format of the operand is an integer format, the second sign and the integer mantissa matching the second multiplication-addition operation factor are parsed; when the input format of the operand is a floating-point format, the second sign, the second exponent, and the second fraction matching the second multiplication-addition operation factor are parsed; regardless of the input format, the sign part takes the highest bit of the input, that is, the second sign is used as the standard sign of the second multiplication-addition operation factor; when the input format is a floating-point format, the fraction part is parsed according to the The value is processed, specifically: when the exponent value is 0 and the fraction value is not 0, the exponent is modified to 1, otherwise, the exponent value is not modified, that is, if the input format for the operand is a floating-point format, then when the second exponent is 0 and the second fraction is not 0, 1 is used as the standard exponent of the second multiplication-addition operation factor; when the second exponent is not 0 or the second fraction is 0, the second exponent is used as the standard exponent of the second multiplication-addition operation factor; when the input format is an integer format, it is processed according to the bit width of the integer format, the exponent bit width of the floating-point format and the input value. When the input value is 0, the exponent value is 0, otherwise, the exponent value is Among them WA I is the bit width when the input format is integer format, WEA F is the exponent width when the input format is a floating-point format, that is, if the input format for the operand is an integer format, when the integer mantissa is 0, 0 is used as the standard exponent of the second multiplication and addition factor; when the integer mantissa is not 0, As the standard exponent of the second multiply-add operation factor, WA I is the bit width in the operand integer format, WEA F The exponent width of the operand's floating-point format.

[0076] The bit width of the mantissa can be determined by taking WA I With WFA F +2, where WFA F Indicates the bit width of the fractional part when the input is in floating point format. The bit width of the mantissa after unpacking is divided into two cases:

[0077] in WA I Greater than or equal to (WEA F+2), if the input format of the operand is a floating-point format, first, fill the highest bit of the fraction with 1 bit of 0, if the exponent part is 0, then fill the integer 0 in the high bit after the highest bit of the fraction part, otherwise fill the integer 1, and fill the last bit with a number of redundant 0s in order to align with the mantissa bit width after unpacking when the input is an integer, that is, when the second exponent is 0, fill the highest bit of the second fraction with 00, fill the lowest bit of the second fraction with 0s of the target number of bits, and use the supplemented result as the standard mantissa of the second multiplication and addition factor; when the second exponent is 1, fill the highest bit of the second fraction with 01, fill the lowest bit of the second fraction with 0s of the target number of bits, and use the supplemented result as the standard mantissa of the second multiplication and addition factor; if the input format of the operand is an integer format, the combination of the second sign and the integer mantissa is used as the standard mantissa of the second multiplication and addition factor.

[0078] in WA I Less than (WEA F +2), if the input format of the operand is a floating point format, first fill the highest bit of the fraction with 1 bit of 0, if the exponent part is 0, then fill the integer 0 after the highest bit of the fraction part, otherwise fill the integer 1, that is, when the second exponent is 0, fill the highest bit of the second fraction with 00, and use the added result as the standard mantissa of the second multiplication and addition operation factor; when the second exponent is 1, fill the highest bit of the second fraction with 01, and use the added result as the standard mantissa of the second multiplication and addition operation factor; if the input format of the operand is an integer format, then fill the combination of the second sign and the integer mantissa with a number of redundant 0s at the end in order to align with the mantissa bit width after unpacking when the input is a floating point, that is, fill the lowest bit of the combination result with 0 of the target number of bits, and use the added result as the standard mantissa of the second multiplication and addition operation factor; where the target number of bits is |(WEA F +2)-WA I |, |.| are absolute value symbols. For example, when the input floating point format is FP32 and the input integer format is INT32, WEA F +2=25,WA I =32, therefore, the number of last 0 digits, i.e. the target number of digits, is 7.

[0079] Based on the above embodiments, the special value detection subunit is specifically used for:

[0080] When the input format of the currently detected multiplication and addition operation factor is a floating point format, if the exponent of the currently detected multiplication and addition operation factor is the maximum value and the fraction is 0, it is determined that the currently detected multiplication and addition operation factor is infinite; if the exponent of the currently detected multiplication and addition operation factor is the maximum value and the fraction is non-zero, it is determined that the currently detected multiplication and addition operation factor is a non-number; if the exponent of the currently detected multiplication and addition operation factor is 0 and the fraction is 0, it is determined that the currently detected multiplication and addition operation factor is 0;

[0081] When the input format of the currently detected multiplication and addition operation factor is an integer format, if the integer mantissa of the currently detected multiplication and addition operation factor is 0, it is determined that the currently detected multiplication and addition operation factor is 0;

[0082] When it is determined that any input multiplication and addition operation factor is a non-number, determining that the output result is a non-number;

[0083] When it is determined that the input operand is 0 and the scaling factor is infinite, or the input operand is infinite and the scaling factor is 0, determining that the output result is a non-number;

[0084] When it is determined that any one of the input operand and the scaling factor is infinite, the offset is infinite and the sign of the offset is opposite to the sign of the product obtained by multiplying the operand by the scaling factor, the output result is a non-number;

[0085] If any of the input operands, scaling factors, and offsets is determined to be infinite, the output result is not a number or infinity.

[0086] Specifically, the special value detection method can be: for a number in a floating point format, when the value of the exponent part is the maximum value and the fraction part is 0, infinity is detected, when the value of the exponent part is the maximum value and the fraction part is non-0, a non-number is detected, and the value of the exponent part is 0 and the value of the fraction part is also 0, zero is detected; for a number in an integer format, when the integer value is 0, zero is detected, and there is no infinity or non-number in the integer format, that is, when the input format of the currently detected multiplication and addition operation factor is a floating point format, if the exponent of the currently detected multiplication and addition operation factor is the maximum value and the fraction is 0, then the currently detected multiplication and addition operation factor is determined to be infinity; if the exponent of the currently detected multiplication and addition operation factor is the maximum value and the fraction is non-0, then the currently detected multiplication and addition operation factor is determined to be a non-number; if the exponent of the currently detected multiplication and addition operation factor is 0 and the fraction is 0, then the currently detected multiplication and addition operation factor is determined to be 0; when the input format of the currently detected multiplication and addition operation factor is an integer format, if the integer mantissa of the currently detected multiplication and addition operation factor is 0, then the currently detected multiplication and addition operation factor is determined to be 0.

[0087] According to the detection results of the input operands, scaling factors and offsets, special values ​​of the results after the multiplication and addition operations can be detected, specifically: when any one of the operands, scaling factors and offsets is a non-number, the result is a non-number; when the operand is zero and the scaling factor is infinite, or the operand is infinite and the scaling factor is zero, the result is a non-number; when any one of the operands and the scaling factor is infinite and the other is not 0, the product of the operand and the scaling factor is infinite, and the offset is infinite, and its sign is opposite to the sign of the product, then the result is a non-number; when any one of the operands, the scaling factor and the offset is infinite, and the result is not a non-number, the result is infinite.

[0088] Based on the above embodiments, the multiplication and addition hardware provided by the embodiments of the present invention can perform on-chip reconstruction of configuration information in real time according to the configuration information of the input format, output format and rounding type that matches the current computing scenario to meet the computing requirements of the current computing scenario.

[0089] Specifically, the multiplication and addition hardware provided by the embodiment of the present invention can dynamically adjust its configuration according to the specific needs of the current computing scenario, according to the set input and output formats and the required rounding type, and automatically reconstruct it into a multiplication and addition unit suitable for the corresponding data format to adapt to different data precisions (such as FP16 or FP32) and rounding modes (such as rounding to the nearest even number or rounding to zero, etc.), so as to optimize computing efficiency and resource utilization, and improve the applicability and flexibility of the hardware in various computing scenarios.

[0090] Embodiment 2

[0091] Figure 2 A schematic diagram of the structure of another mixed-precision multiplication and addition hardware provided in Embodiment 2 of the present invention. This embodiment is a refinement of the floating-point multiplier and adder in the above-mentioned embodiment, and specifically includes: an exponent calculation subunit, an addend order subunit, a mantissa multiplication subunit, and a summation subunit.

[0092] Correspondingly, such as Figure 2 As shown, the mixed-precision multiplication-addition hardware includes: an input processing unit 210, an exponent calculation subunit 220, an addend order subunit 230, a mantissa multiplication subunit 240, a summation subunit 250 and an output processing unit 260.

[0093] The input processing unit 210 is used to unpack each input multiplication and addition operation factor according to a pre-configured input format, obtain a standard sign, a standard exponent and a standard mantissa corresponding to each multiplication and addition operation factor, and obtain an input and output special value detection result;

[0094] The exponent calculation subunit 220 is used to calculate the product exponent of the operand multiplied by the scaling factor, the exponential difference between the product and the offset, the order shift value of the offset, and the multiplication and addition result exponent of the product plus the offset according to the standard exponent of each multiplication and addition operation factor;

[0095] The addend alignment subunit 230 is used to perform two's complement and exponent alignment on the standard mantissa of the offset to obtain an aligned signed addend;

[0096] The mantissa multiplication subunit 240 is used to perform signed multiplication according to the standard mantissa of the operand and the scaling factor to obtain a signed product;

[0097] The summing subunit 250 is used to sum the signed product and the signed addend to obtain a multiplication-addition result mantissa, wherein the multiplication-addition result exponent and the multiplication-addition result mantissa are combined to form a floating-point multiplication-addition operation result.

[0098] The output processing unit 260 is used to normalize and round the floating-point multiplication and addition operation results according to the pre-configured output format and rounding type to obtain the target format result, and to package and output the target format result according to the input and output special value detection result;

[0099] Among them, the types of multiplication and addition operation factors include: operands, scaling factors and offsets, the types of input formats and output formats include floating-point formats or integer formats with set bit widths of each multiplication and addition operation factor, and the bit width of the floating-point format includes: exponent bit width and fraction bit width.

[0100] Specifically, the input processing unit 210 performs unpacking processing on each input multiplication and addition operation factor according to the pre-configured input format, obtains the standard sign, standard exponent and standard mantissa corresponding to each multiplication and addition operation factor, and obtains the input and output special value detection result. The exponent calculation subunit 220 calculates the product exponent of the operand multiplied by the scaling factor, the exponential difference between the product and the offset, the offset order shift value and the product plus the offset multiplication and addition result exponent according to the standard exponent of each multiplication and addition operation factor. The addend order subunit 230 performs the complement code and exponent order processing on the standard mantissa of the offset to obtain the signed addend after order. The mantissa multiplication subunit 240 performs signed multiplication according to the standard mantissa of the operand and the scaling factor to obtain a signed product, such as: extracting the standard mantissa part from each operand, and then performing binary multiplication according to the sign bit of the operand to ensure that the sign of the result is correct. The signed product and the signed addend are summed by the summing subunit 250 to obtain a multiplication-addition result mantissa, wherein the multiplication-addition result exponent and the multiplication-addition result mantissa are combined to form a floating-point multiplication-addition operation result. The floating-point multiplication-addition operation result is normalized and rounded by the output processing unit 260 according to the pre-configured output format and rounding type to obtain a target format result, and the target format result is packaged and output according to the input and output special value detection result.

[0101] The mixed-precision multiplication and addition hardware of the embodiment of the present invention includes: an input processing unit, an exponent calculation subunit, an addend order subunit, a mantissa multiplication subunit, a summation subunit and an output processing unit, wherein: the input processing unit is used to unpack each input multiplication and addition operation factor according to a pre-configured input format, obtain the corresponding standard symbol, standard exponent and standard mantissa, and obtain the input and output special value detection results; the exponent calculation subunit is used to calculate the product exponent of the operand multiplied by the scaling factor, the exponent difference between the product and the offset, the order shift value of the offset and the product addend according to the standard exponent of each multiplication and addition operation factor. The multiplication and addition result exponent of the upper offset; the addend order subunit is used to take the complement code and order the exponent of the standard mantissa of the offset to obtain the signed addend after ordering; the mantissa multiplication subunit is used to perform signed multiplication according to the standard mantissa of the operand and the scaling factor to obtain the signed product; the summation subunit is used to sum the signed product and the signed addend to obtain the mantissa of the multiplication and addition result; the output processing unit is used to normalize and round the floating-point multiplication and addition operation result according to the pre-configured output format and rounding type to obtain the target format result, and pack and output the target format result according to the input and output special value detection result. Through the exponent calculation subunit, addend order subunit, mantissa multiplication subunit and summation subunit, the floating-point multiplier and adder supports the input operands in the intermediate format generated by the input processing unit conversion, realizes the reuse of the data path of the floating-point multiplier and adder, saves chip area, integrates format conversion, simplifies data flow, realizes the automatic reconstruction of the multiplication and addition operation unit according to the input and output formats, and the requirements of quantization and dequantization of floating points and integers, and improves the reuse rate of computing resources.

[0102] Based on the above embodiments, the mantissa multiplication subunit 240 is specifically used for:

[0103] Among the standard mantissas of the operand and the scaling factor, a standard mantissa with a smaller bit width is selected as a first multiplier, and another standard mantissa is selected as a second multiplier;

[0104] Perform booth encoding on the first multiplier to obtain an encoding result;

[0105] If the input format of the operand is a floating point format, the operand and the standard sign of the scaling factor are XORed, and when the XOR value is 1, the encoding result is inverted to obtain an updated encoding result;

[0106] If the input format of the operand is an integer format, when the standard sign of the scaling factor is 1, the encoding result is inverted to obtain an updated encoding result;

[0107] A partial product is generated according to the current encoding result and the second multiplier, and the partial product is compressed using a tree adder based on a carry-save adder to obtain the signed product.

[0108] Specifically, booth encoding is performed on the mantissas with smaller bit widths unpacked from the operands and scaling factors, that is, among the standard mantissas of the operands and scaling factors, a standard mantissa with smaller bit width is selected as the first multiplier, and another standard mantissa is used as the second multiplier. The first multiplier is booth encoded to obtain an encoded result. The first and second here are only used to distinguish the multipliers, and there is no order of precedence. Booth encoding is used to reduce the number of partial products and speed up the mantissa multiplication operation.

[0109] After the booth code is generated, it is also necessary to decide whether to post-process the booth code according to the signs of the operands and scaling factors unpacked. Specifically, when the operand is in floating-point format, if the signs of the operands and scaling factors unpacked are 1 after XOR, the booth code needs to be inverted, otherwise it is not necessary; when the operand is in integer format, if the sign of the scaling factor unpacked is 1, the booth code needs to be inverted, otherwise it is not necessary. That is, if the input format of the operand is floating-point format, the standard signs of the operands and scaling factors are XORed, and when the XOR value is 1, the coding result is inverted to obtain an updated coding result; if the input format of the operand is integer format, when the standard sign of the scaling factor is 1, the coding result is inverted to obtain an updated coding result.

[0110] The value with a larger bit width in the mantissa unpacked from the operand and the scaling factor is used as another multiplier, and a partial product is generated based on it using booth coding through post-processing, and the partial product is compressed using a tree adder based on a carry-preserve adder to obtain a signed product, that is, a partial product is generated according to the current coding result and the second multiplier, and the partial product is compressed using a tree adder based on a carry-preserve adder to obtain a signed product. The booth coding is used to reduce the number of partial products to speed up the multiplication speed, and combined with the tree adder, the performance of the multiplier can be further improved.

[0111] Embodiment 3

[0112] Figure 3 A structural diagram of another mixed-precision multiplication-addition hardware provided in Embodiment 3 of the present invention. This embodiment is a refinement of the output processing unit in the above-mentioned embodiment, and specifically includes: a multi-precision normalization subunit, a rounding subunit, and a result packing subunit.

[0113] Correspondingly, such as Figure 3As shown, the mixed-precision multiplication-addition operation hardware includes: an input processing unit 310, a floating-point multiplier-adder 320, a multi-precision normalization subunit 330, a rounding subunit 340 and a result packing subunit 350.

[0114] The input processing unit 310 is used to unpack each input multiplication and addition operation factor according to a pre-configured input format, obtain a standard sign, a standard exponent and a standard mantissa corresponding to each multiplication and addition operation factor, and obtain an input and output special value detection result;

[0115] A floating-point multiplier-adder 320, for calculating the floating-point multiplication-addition result obtained by multiplying the operand by the scaling factor and accumulating the offset according to the standard sign, standard exponent and standard mantissa of each multiplication-addition operation factor;

[0116] The multi-precision normalization subunit 330 is used to determine the shift value during normalization according to the pre-configured output format, and perform normalization processing on the floating-point multiplication and addition operation result according to the shift value to obtain the normalized operation result;

[0117] The rounding subunit 340 is used to determine the carry value of the normalized operation result according to the pre-configured rounding type, and calculate the rounded multiplication-addition result mantissa and the corrected multiplication-addition result exponent according to the carry value;

[0118] The result packing subunit 350 is used to generate a target format result according to the output format, the rounded multiplication-addition result mantissa, the corrected multiplication-addition result exponent, and the input and output special value detection result, and to pack and output the target format result.

[0119] Specifically, the input processing unit 310 performs unpacking processing on each input multiplication and addition operation factor according to the pre-configured input format, obtains the standard sign, standard exponent and standard mantissa corresponding to each multiplication and addition operation factor, and obtains the input and output special value detection result. The floating-point multiplier 320 calculates the floating-point multiplication and addition operation result obtained by multiplying the operand by the scaling factor and accumulating the offset according to the standard sign, standard exponent and standard mantissa of each multiplication and addition operation factor. The multi-precision normalization subunit 330 identifies the most significant bit of the result mantissa, and calculates the number of bits that need to be moved to the left or right of the mantissa so that the most significant bit is located at the desired position, thereby meeting the output format requirements for the number of mantissa bits.

[0120] When performing normalization processing, the mantissa is moved according to the calculated shift value, and the exponent value is adjusted accordingly to obtain a normalized floating-point result, that is, the shift value during normalization is determined according to the pre-configured output format, and the floating-point multiplication and addition operation result is normalized according to the shift value to obtain a normalized operation result. The rounding subunit 340 determines the carry value of the normalized operation result according to a pre-set rounding type, such as nearest even rounding or rounding to zero, and can determine whether it is necessary to carry to a higher bit by checking a specific bit of the mantissa (such as the operation result of dividing the signed valid value), that is, the carry value of the normalized operation result is determined according to the pre-configured rounding type, and then the mantissa of the multiplication and addition result after rounding and the exponent of the multiplication and addition result after correction are calculated according to the carry value.

[0121] The result packing subunit 350 generates the target format result according to the output format, the rounded multiplication and addition result mantissa, the corrected multiplication and addition result exponent, and the input and output special value detection result. Specifically, when the output format is an integer, the rounded multiplication and addition result mantissa and the corrected multiplication and addition result exponent obtained by the rounding subunit 340 are obtained, and the rounded multiplication and addition result mantissa is rounded to the corrected multiplication and addition result exponent. A preset rounding mode such as rounding to the nearest even number or rounding to zero can be used to convert the rounded floating point number to an integer and discard the decimal part. When the output format is a floating point number, the specific format requirements are first determined, such as single precision or double precision, and then the mantissa of the multiplication and addition result is rounded according to the preset rounding type, and the exponent is corrected to ensure that the combination of the mantissa and the exponent meets the requirements of the floating point format. If the input or intermediate result contains special values, such as NaN or infinity, they are encoded according to the IEEE 754 standard. For example, the sign bit indicates positive or negative, the sign bit of positive infinity is 0, and the sign bit of negative infinity is 1. The exponent bits are all set to 1 to indicate the maximum value of the exponent, and the mantissa bits are all set to 0. For NaN, the exponent bits are all set to 1, and the mantissa bits are not all 0. At least one bit is 1.

[0122] Then, the target format result is packaged and output. The output bit width is determined by the maximum bit width of all supported output formats. For example, when the output format supports FP32, FP16, BF16, INT32, and INT16, the output bit width is 32 bits. If the output bit width is greater than the bit width of the output format, the excess high bits are filled with 0.

[0123] The mixed-precision multiplication and addition hardware of the embodiment of the present invention comprises: an input processing unit, a floating-point multiplier and adder, and an output processing unit, wherein: the input processing unit is used to unpack each input multiplication and addition operation factor according to a pre-configured input format, obtain the corresponding standard sign, standard exponent and standard mantissa, and obtain the input and output special value detection result; the floating-point multiplier and adder is used to calculate the floating-point multiplication and addition operation result obtained by multiplying the operand by the scaling factor and accumulating the offset according to the standard sign, standard exponent and standard mantissa of each multiplication and addition operation factor; the multi-precision normalization subunit, It is used to determine the shift value during normalization according to the pre-configured output format, and normalize the floating-point multiplication and addition operation result according to the shift value to obtain the normalized operation result; the rounding subunit is used to determine the carry value of the normalized operation result according to the pre-configured rounding type, and calculate the rounded multiplication and addition result mantissa and the corrected multiplication and addition result exponent according to the carry value; the result packing subunit is used to generate the target format result according to the output format, the rounded multiplication and addition result mantissa, the corrected multiplication and addition result exponent and the input and output special value detection result, and pack and output the target format result. Through the normalization, rounding and packing output subunits, the output processing unit supports multiple floating-point or integer formats, integrates format conversion, simplifies data flow, and can convert full-precision floating-point multiplication and addition results into any output format. By reusing the data path of the floating-point multiplier and adder, the chip area is saved, and the automatic reconstruction of the multiplication and addition operation unit according to the input and output format and the requirements for quantization and dequantization of floating points and integers are realized, which improves the reuse rate of computing resources.

[0124] Based on the above embodiments, the multi-precision normalization subunit 330 is specifically used for:

[0125] Perform redundant sign bit detection on the mantissa of the multiplication and addition result to obtain the redundant sign bit number, and determine the sign bit padding number according to the maximum bit width of all integer formats and the maximum fractional bit width of all floating point formats that can be supported by the output format;

[0126] Supplement the high bit of the mantissa of the multiplication-addition result with a sign bit that matches the number of sign bit padding to obtain the mantissa of the multiplication-addition result after padding;

[0127] Determine the number of decimal point left shifts after normalization according to the output format, and modify the multiplication-addition result exponent to the corrected multiplication-addition result exponent according to the number of decimal point left shifts;

[0128] Calculate the number of left shifts corresponding to the mantissa of the multiplication-addition result according to the number of redundant sign bits, the multiplication-addition result exponent, and the output format;

[0129] According to the number of left shifts, the mantissa of the padded multiplication-addition result is left-shifted and normalized to obtain a normalized multiplication-addition result mantissa, and the current corrected multiplication-addition result exponent is modified to a new corrected multiplication-addition result exponent that matches the number of left shifts.

[0130] Specifically, before the normalization operation, the number of redundant sign bits of the input multiplication and addition result mantissa is first detected. The detection circuit is a typical circuit, such as a look-ahead encoder or a tree detector, and the number of redundant sign bits obtained is LSC. While detecting the redundant sign bits, it is necessary to fill the high bits of the mantissa of the multiplication and addition result with a number of sign bits to avoid the problem that when the output format is an integer, it may be necessary to use left and right shifts in two directions. Specifically, the number of sign bit fills can be determined based on the maximum bit width of all integer formats supported by the output format and the maximum fractional bit width of all floating-point formats. The integer formats supported by the output are INT32 and INT16, and the floating-point formats supported are FP32, FP16, and BF16. The number of bits that need to be filled is WZ norm For WZI max +1=33 and WZFF max +3 = the maximum value among 26, that is, 33 bits, where WZ norm is the number of digits to be filled, WZI max Indicates the maximum bit width in the integer format supported by the output, WZFF max Indicates the maximum fractional bit width in the floating point format supported by the output. The high bit of the multiplication-addition result mantissa is supplemented with a sign bit that matches the sign bit supplementation number to obtain the mantissa of the multiplication-addition result after supplementation.

[0131] During normalization, the number of decimal point left shifts after normalization is determined according to the output format. For integer format, the decimal point position is WZI+1 bits before normalization, where WZI represents the bit width of the output integer format; for floating-point format, the decimal point position is WZFF+3 bits before normalization, where WZFF represents the fractional bit width of the output floating-point format. Then, according to the number of decimal point left shifts, the multiplication and addition result exponent is modified to the corrected multiplication and addition result exponent, that is, when the decimal point is shifted to the left, the exponent value needs to be modified according to the number of bits shifted to the left. Let the exponent value before the decimal point is shifted to the left be E0, and the exponent value at the decimal point after the left shift be E1=E0+LSH, where LSH represents the number of bits shifted to the left. For example, for INT32, LSH=WZI+1, and for FP32, LSH=WZFF+3.

[0132] Then, according to the number of redundant sign bits, the exponent of the multiplication and addition result, and the output format, the number of left shift bits corresponding to the mantissa of the multiplication and addition result is calculated. When the output format is an integer format, the formula for calculating the number of left shift bits PLSH is:

[0133]

[0134] When the output format is floating point format, the formula for calculating the left shift bit PLSH is:

[0135]

[0136] Use the above two formulas to calculate the number of left shifts, perform left shift and normalization processing on the mantissa of the multiplication and addition result after padding, obtain the normalized mantissa of the multiplication and addition result, and modify the current corrected multiplication and addition result exponent to a new corrected multiplication and addition result exponent that matches the number of left shifts, that is, the exponent value E1 needs to be modified at the same time, and obtain E2=E1-PLSH.

[0137] Based on the above embodiments, the rounding subunit 340 is specifically used for:

[0138] After the integer 0 is added before the highest bit of the mantissa of the normalized multiplication and addition result, the valid value part and the discarded value part are identified in the mantissa of the normalized multiplication and addition result after the 0 is added;

[0139] Generate a carry value according to the rounding mode and the discarded value part, and calculate the sum of the effective value part and the carry value as the mantissa of the multiplication and addition result after rounding;

[0140] If the output format is a floating point format, the absolute value of the rounded multiplication-addition result mantissa is calculated as the new rounded multiplication-addition result mantissa, and the current corrected multiplication-addition result exponent is modified again according to the fractional bit width of the new rounded multiplication-addition result mantissa to obtain a new corrected multiplication-addition result exponent.

[0141] Specifically, after a 1-bit integer 0 is added to the highest bit of the normalized multiplication and addition result mantissa, the valid value part and the discarded value part are identified in the normalized multiplication and addition result mantissa after the 0 is added, wherein the valid value part can be WZ in the normalized multiplication and addition result mantissa. norm The high-order value of the bit, the rest of the normalized multiplication and addition result mantissa is the discarded value part, WZ normThe required complement number calculated in the multi-precision normalization subunit 330. According to the rounding mode and the discarded value part, the carry value can also be combined with the lowest bit of the effective value to generate a carry value, wherein the carry value can only be 1 or 0. Then the sum of the effective value part and the carry value is calculated as the mantissa of the rounded multiplication and addition result. If the output format is a floating point format, the absolute value of the mantissa of the rounded multiplication and addition result is calculated as the new mantissa of the rounded multiplication and addition result, and according to the fractional bit width of the new mantissa of the rounded multiplication and addition result, the current corrected multiplication and addition result exponent is modified again to obtain a new corrected multiplication and addition result exponent, which can be specifically: according to the sign bit of the multiplication and addition result, its absolute value is calculated, and the sign bit is recorded as SZ. Afterwards, according to the output format and the corrected multiplication and addition result exponent E2, it is judged whether the WZFF+1 bit or the WZFF bit of the absolute value of the rounded multiplication and addition result is 1, and it is determined whether E2 is corrected accordingly. Wherein, WZFF represents the fractional bit width of the output floating point format.

[0142] In a specific example, when the output format is FP16, WZFF=11. When E2 is 0, it is determined whether the WZFF bit is 1. If so, E2 needs to be added by 1, otherwise not. When E2 is not 0, it is determined whether the WZFF+1 bit is 1. If so, E2 needs to be added by 1, otherwise not, thereby obtaining a new corrected multiplication and addition result exponent E3.

[0143] Based on the above embodiments, the result packaging subunit 350 is specifically used for:

[0144] If the output format is an integer format, when the corrected multiplication-addition result exponent is greater than the bit width of the integer format -2, or when the corrected multiplication-addition result exponent is equal to the bit width of the integer format -2 and the bit width bit of the integer format in the rounded multiplication-addition result mantissa is different from the value of the bit width bit -1 of the integer format, the mantissa of the rounded multiplication-addition result is saturated to the maximum value of the corresponding sign as the output result; otherwise, the low-order data matching the bit width of the integer format is intercepted in the mantissa of the rounded multiplication-addition result as the output result;

[0145] If the output format is a floating point format, then when the input and output special value detection result is valid, the matching output special value detection result is output; otherwise, it is determined whether the corrected multiplication and addition result exponent exceeds the representation range of the normalized number in the floating point format;

[0146] If it exceeds, the result is rounded to infinity or the maximum normalized value according to the rounding mode; if it does not exceed, the mantissa of the multiplication and addition result after rounding is determined to be 0;

[0147] If it is 0, output 0; if it is not 0, when the corrected multiplication and addition result exponent is less than 0, the corrected multiplication and addition result exponent is normalized to 0 and then the matching output result is output; when the corrected multiplication and addition result exponent is greater than or equal to 0, the matching output result is directly output.

[0148] Specifically, if the output format is an integer format, first detect the relationship between the corrected multiplication and addition result exponent E3 and WZI-2, where WZI is the bit width of the integer format. If E3 is greater than WZI-2, or if E3 is equal to WZI-2 and the values ​​of the WZI-1th bit of the multiplication and addition result after rounding are different, saturate the mantissa of the multiplication and addition result after rounding to the maximum value of the corresponding sign as the output result, otherwise, intercept the low-order data matching the bit width of the integer format in the mantissa of the multiplication and addition result after rounding as the output result. Saturation means limiting the calculation result to a predefined numerical range to prevent overflow or underflow, that is, setting the calculation result out of the range to the maximum value or minimum value within the range.

[0149] If the output format is a floating-point format, first determine whether the input special result test value is valid. When the input and output special value test result is valid, output the matching output special value test result; otherwise, determine whether E3 exceeds the representation range of the normalized number in the floating-point format. If it exceeds, round the result to infinity or the maximum normalized value according to the rounding mode; if it does not exceed, determine whether the rounded multiplication and addition result mantissa is 0; if it is 0, output 0; if it is not 0, then when the corrected multiplication and addition result exponent is less than 0, normalize E3 to 0, and output the matching output result according to the absolute value of the multiplication and addition result; when the corrected multiplication and addition result exponent is greater than or equal to 0, directly output the matching output result according to the exponent value and the absolute value of the multiplication and addition result.

[0150] Specific application scenarios

[0151] The current mainstream deep learning algorithms generally support mixed-precision training and inference modes. In order to improve hardware throughput and computing power, low-precision data formats (such as FP8 and INT8, etc.) are used for input and output, and high-precision data formats (such as FP32 and FP64, etc.) are used for internal accumulation. When using low-precision data formats as output, the output needs to be dequantized and quantized, that is, the multiplication and addition operation of Z = A*Scale+Bias, where Z is the output, A is the input, Scale is the scaling factor, and Bias is the offset. However, the input and output formats of input A and output Z can be floating-point format and integer format. The usual technical solution is implemented using independent data paths. In floating-point mode, floating-point data paths are used for calculation, and in integer mode, integer data paths are used for calculation, resulting in idle computing resources, which is not conducive to reducing chip area. To solve the above problems, an embodiment of the present invention proposes a mixed-precision multiplication and addition hardware. Figure 4 is a flowchart of a multi-precision multiplication and addition operation applicable to the embodiment of the present invention. Figure 4 As shown, the process of multi-precision multiplication and addition operation may specifically include:

[0152] 1. Input processing: Unpack each input operand according to the configuration, split it into three parts: sign, exponent and mantissa, and detect whether the input operand contains three special values: 0, infinity and non-number, and detect whether the output result is infinity and non-number. After input processing, the input operands in floating point / integer format can be converted into a unified intermediate format for subsequent calculations.

[0153] 2. Floating-point multiplication and addition: According to the split results, floating-point multiplication and addition operations are performed to obtain full-precision floating-point multiplication and addition results, which include the exponent part and the mantissa part. In addition, the special value detection results of the input are directly passed to the output.

[0154] 3. Output processing, which is used to perform multi-precision normalization and rounding on the results of floating-point multiplication and addition operations, and to package and output the results according to the results of special value detection and output precision.

[0155] By using input processing, input operands of different formats are converted into a unified intermediate format to reuse the floating-point multipliers that occupy most of the area. By using output processing, full-precision floating-point multiplication and addition results can be converted into any output format, thus realizing format conversion and simplifying the processing logic.

[0156] Figure 5 is a flow chart of a mixed-precision multiplication-addition operation applicable to an embodiment of the present invention. Figure 5 As shown:

[0157] The input operand A supports two data formats, one floating-point data format and one integer data format. The bit width of the input operand A is the maximum value of the bit width of the floating-point data format and the integer data format. For example, if the floating-point data format is FP16 and the integer data format is INT32, the bit width of the input operand is 32 bits. In this case, Figure 6 is a schematic diagram of a preferred input data arrangement scheme applicable to an embodiment of the present invention, such as Figure 6 As shown, the floating-point data format is FP32, the integer data format is INT32, and the bit width of the input operand is 32 bits.

[0158] The input format controls the data format of the input operand A and is used to identify whether the input operand A is in floating-point data format or integer data format.

[0159] The scaling factor Scale and the offset Bias only support one floating point data format. Preferably, the floating point data formats of Scale and Bias are the same, and can both be FP32 format.

[0160] The rounding mode controls the rounding mode used during the operation, and supports setting multiple rounding modes, including but not limited to rounding towards 0, rounding towards positive infinity, rounding towards negative infinity, and rounding towards the nearest even number. Preferably, the mixed-precision multiplication and addition hardware supports the above four rounding modes.

[0161] The output format controls the data format of the output result Z, including multiple floating point data formats and multiple integer data formats. Preferably, the floating point data formats supported by the output are FP32, FP16 and BiasF16 (16-bit half-precision floating point numbers), and the integer data formats supported by the output are INT32 and INT16.

[0162] The bit width of the output result Z is determined by the maximum bit width of all supported output formats. For example, when the output format supports FP32, FP16, BF16, INT32, and INT16, the output bit width is 32 bits. Figure 7 is a schematic diagram of a preferred output result arrangement scheme applicable to an embodiment of the present invention. If the output bit width is greater than the output format bit width ( Figure 7 32 bits is used as an example), the valid data is located in the low bits (the low 16 bits in FP16), and the extra high bits are filled with 0, such as Figure 7 shown.

[0163] Specifically, the input processing unit in the mixed-precision multiplication-addition hardware includes: a mixed-precision unpacking and special value detection module, which unpacks and performs special value detection on the input operand A, the scaling factor Scale, and the offset Bias, and detects special multiplication-addition results.

[0164] Figure 8 is a schematic diagram of a floating point data format applicable to an embodiment of the present invention, such as Figure 8 As shown, the unpacking method for the scaling factor Scale and the offset Bias is as follows: the scaling factor Scale and the shift amount are in the IEEE 754 standard floating point format, and the format includes three parts: a sign (S), an exponent (E), and a fraction (F). Fig. 9 is a schematic diagram of an unpacking result applicable to an embodiment of the present invention, such as Fig. 9 As shown, when unpacking:

[0165] The symbol part does not require special processing and can be output directly.

[0166] The exponent part needs to be processed according to the value of the fraction part. The specific logic is: when the exponent value is 0 and the fraction value is not 0, the exponent is modified to 1, otherwise, the exponent value is not modified.

[0167] The mantissa is formed based on the values ​​of the exponent and fraction. The specific logic is: if the exponent is 0, the highest bit of the fraction needs to be padded with an integer 0, otherwise padded with an integer 1. The final mantissa is shown in the following figure, where "I" represents the complemented integer and "F" represents the fraction in the input operand.

[0168] The unpacking method of the input operand A varies according to the input format. When unpacking:

[0169] Regardless of the input format, the sign part takes the highest bit of the input.

[0170] Exponent part. When the input format is a floating point format, it is processed according to the value of the fraction part. The specific logic is: when the exponent value is 0 and the fraction value is not 0, the exponent is modified to 1, otherwise, the exponent value is not modified. When the input format is an integer format, it is processed according to the bit width of the integer format, the exponent bit width of the floating point format and the input value. When the input value is 0, the exponent value is 0, otherwise, the exponent value is Among them WA I is the bit width when the input format is integer format, WEA F The exponent width when the input format is floating point format.

[0171] The mantissa part. The bit width of the mantissa part is determined by the following rules: Take WA I With WFA F +2, where WFA F Indicates the bit width of the fractional part when the input is in floating point format. The bit width of the mantissa after unpacking is divided into two cases:

[0172] (1) Take WA I , Fig.10 is a schematic diagram of a mantissa applicable to an embodiment of the present invention. In this case, the mantissa after unpacking is as follows Fig.10 As shown in the figure, the first "0" is 1-bit 0, and "I" indicates the complemented integer. If the exponent part is 0, the integer 0 needs to be added to the highest bit after the highest bit of the fraction part, otherwise the integer 1 is added. "F" indicates the fraction part in the floating-point format, and "0" indicates that several redundant 0s are added to the last bit to align with the mantissa bit width after unpacking when the input is an integer. "INT" indicates the complete integer value when the input is an integer, including the sign bit.

[0173] (2) Take WFA F +2, Fig.11 is a schematic diagram of another mantissa applicable to the embodiment of the present invention. In this case, the mantissa after unpacking is as follows Fig.11 As shown in the figure, the first "0" is 1-bit 0, and "I" indicates the integer to be padded. If the exponent part is 0, the integer 0 needs to be padded after the highest bit of the fraction part, otherwise the integer 1 is padded. "F" indicates the fraction part in the floating point format. "INT" indicates the complete integer value when the input is an integer, including the sign bit. "0" indicates that several redundant 0s are padded at the end to align with the mantissa bit width after unpacking when the input is a floating point.

[0174] The calculation method of the number of 0 bits at the end is as follows: First, determine the width of the mantissa after unpacking WMO = max (WFA F +2,WA I ), then take WMO-WFA F +2 and WMO-WA I For example, when the input floating point format is FP32 and the input integer format is INT32, WFA F +2=25,WA I =32, therefore, use the larger value of the two, WMO=32, and the number of last 0 bits is 7.

[0175] The process of special value detection is as follows: for numbers in floating point format, when the value of the exponent part is the maximum value and the fraction part is 0, infinity is detected; when the value of the exponent part is the maximum value and the fraction part is not 0, non-number is detected; when the value of the exponent part is 0 and the value of the fraction part is also 0, zero is detected. For numbers in integer format, when the integer value is 0, zero is detected, and there is no infinity or non-number in the integer format.

[0176] Based on the detection results of the input operands A, Scale and Bias, the special value of the result after the multiplication and addition operation can be preliminarily detected. Specifically, it can be:

[0177] When any one of A, Scale and Bias is a non-number, the result is a non-number; when A is zero and Scale is infinite, or A is infinite and Scale is zero, the result is a non-number; when any one of A and Scale is infinite and the other is not 0, the product of A and Scale is infinite, and Bias is infinite, and its sign is opposite to that of the product, then the result is a non-number; when any one of A, Scale and Bias is infinite and the result is not a non-number, the result is infinite.

[0178] The floating-point multiplier-adder in the mixed-precision multiplication-addition hardware proposed in the embodiment of the present invention may specifically include: an exponent calculation module, an addend order module, a mantissa multiplication module and a summation module.

[0179] (1) An exponential calculation module, used to calculate the exponent of the product, the difference between the exponents of the product and the addend, the order shift value of the addend, and the exponent of the multiplication and addition result.

[0180] (2) An addend alignment module is used to perform two’s complement and exponent alignment on the mantissa of the addend (i.e., Bias) in the multiplication-addition operation to obtain a signed addend after alignment.

[0181] (3) A mantissa multiplication module, which is used to perform signed multiplication on the mantissas of two multipliers (i.e., A and Scale) in a multiplication-addition operation.

[0182] Preferably, the multiplier can use booth coding combined with the structure of a tree adder. First, booth coding is performed on the mantissas with smaller bit widths among the mantissas unpacked from A and Scale. This can reduce the number of partial products and speed up the mantissa multiplication operation. In a specific example, the mantissa bit width unpacked from A is 33 bits, and the mantissa bit width unpacked from Scale is 24. Therefore, booth coding is always performed on the mantissa unpacked from Scale.

[0183] After the booth code is generated, it is necessary to decide whether to post-process the booth code according to the symbols unpacked from A and Scale. When A is in floating point format, if the XOR of the symbols unpacked from A and Scale is 1, the booth code needs to be inverted; otherwise, it is not necessary; when A is in integer format, if the symbol unpacked from Scale is 1, the booth code needs to be inverted; otherwise, it is not necessary.

[0184] A larger value in the mantissa unpacked from A and Scale is used as another multiplier, a partial product is generated based on it using booth encoding through post-processing, and the partial product is compressed using a tree adder based on a carry-save adder to obtain a signed product.

[0185] (4) A summation module is used to sum the product and the multiplied addend.

[0186] Fig.12 is a schematic diagram of a bit width when summing that is applicable to an embodiment of the present invention, such as Fig.12 As shown, it is both the bit width when summing and the bit width of the signed order result generated after the addends are ordered. The total bit width is: WMO+WFS+2*WFB+10 bits, where WFS is the bit width of the fraction of Scale and WFB is the bit width of the fraction of Bias. In the preferred implementation, WMO is 32, WFS and WFB are both 23, so the total bit width is 111 bits. Fig.13 is a schematic diagram of a bit arrangement of a product applicable to an embodiment of the present invention, such as Fig.13As shown, the calculated signed product (Prod) is located in the middle (WMO+WFS+2) bits, the high bits are filled with the sign bit of the product, and the low bits are filled with 0s.

[0187] The output processing unit in the mixed-precision multiplication-addition hardware proposed in the embodiment of the present invention may specifically include: a multi-precision normalization module, a rounding module and a result packing module.

[0188] (1) A multi-precision normalization module determines the shift value during normalization according to the set output format and normalizes the result after floating-point multiplication and addition.

[0189] Before normalization, the number of redundant sign bits of the input floating-point multiplication and addition results is first detected. The detection circuit is a typical circuit, such as a look-ahead encoder, a parity checker, a tree adder or a sign bit detector, and the detected number of redundant sign bits is LSC.

[0190] While detecting redundant sign bits, it is necessary to fill in several sign bits in the high order of the floating-point multiplication and addition result to avoid the problem that when the output format is an integer, it may be necessary to use shifts in both the left and right directions. The specific number of bits that need to be filled is determined by the maximum bit width in the integer format supported by the output and the maximum fractional bit width in the floating-point format supported by the output. In the preferred implementation, the integer formats supported by the output are INT32 and INT16, and the floating-point formats supported are FP32, FP16, and BF16. The number of bits that need to be filled is WZ norm For WZI max +1=33 and WZFF max +3 = the maximum value among 26, that is, 33 bits, where WZI max Indicates the maximum bit width in the integer format supported by the output, WZFF max Indicates the maximum fractional bit width in the floating-point format supported by the output. Fig.14 is a schematic diagram of a multiplication and addition result with a high-bit padded sign bit applicable to an embodiment of the present invention, such as Fig.14 As shown, the total bit width becomes WMO+WFS+2*WFB+WZ norm + 10. In a specific example, WMO is 32 bits, WFS and WFB are both 23 bits, and the total bit width is 144 bits. Fig.15 is a schematic diagram of the decimal point position of a floating-point multiplication and addition result applicable to an embodiment of the present invention. After the high-order sign bit is filled, the decimal point position of the floating-point multiplication and addition result is as follows: Fig.15 shown.

[0191] During normalization, the position of the decimal point after normalization is determined according to the output format. For integer format, the decimal point position is the position before normalization shifted left by WZI+1 bits, where WZI represents the bit width of the output integer format; for floating-point format, the decimal point position is the position before normalization shifted left by WZFF+3 bits, where WZFF represents the fractional bit width of the output floating-point format.

[0192] For example, when the output format is INT32, Fig.16 is a schematic diagram of a normalized decimal point position applicable to an embodiment of the present invention. The normalized decimal point position is as follows: Fig.16 shown. Fig.17 is a schematic diagram of another normalized decimal point position applicable to an embodiment of the present invention. When the output format is FP32, the normalized decimal point position is as follows: Fig.17 shown.

[0193] When the decimal point is shifted to the left, the exponent value needs to be modified according to the number of digits shifted to the left. Suppose the exponent value before the decimal point is shifted to the left is E0, and the exponent value at the decimal point after the left shift is E1=E0+LSH, where LSH represents the number of digits shifted to the left. For example, for INT32, LSH=WZI+1, and for FP32, LSH=WZFF+3.

[0194] After determining the exponent value at the decimal point after normalization, the multiplication and addition result after filling the high-order sign bit is shifted left as appropriate. When the output format is integer format, the formula for calculating the left shift bit PLSH is:

[0195]

[0196] When the output format is floating point format, the formula for calculating the left shift bit PLSH is:

[0197]

[0198] Using the above two formulas, the multiplication and addition results after the high-order sign bit is padded are left-shifted for normalization, and the exponent value E1 needs to be modified to obtain E2=E1-PLSH.

[0199] (2) A rounding module, used to determine the carry value of the normalized mantissa according to the rounding mode and calculate the mantissa after rounding.

[0200] Fig.18 : is a schematic diagram of the arrangement of the bits of the normalized multiplication and addition result in a rounding module applicable to an embodiment of the present invention. A 0 is added to the highest bit of the normalized multiplication and addition result to form Fig.18 The total bit width is WMO+WFS+2*WFB+WZ. norm+11 bits. The highest bit is the added S, which is the sign bit, and it is the same as the following WZ norm The -1 bit (valid+1) forms the valid value of the multiplication and addition result after rounding, and the following WMO+WFS+2*WFB+11 bits are the parts that are discarded after rounding, namely sbits and rbit.

[0201] When rounding, a carry value is generated according to the rounding mode and the lbit, rbit, and sbits in the figure. The carry value can only be 1 or 0. Then the carry value is added to the value of the valid part (signed valid data) in the figure to get the rounded multiplication and addition result. Fig.19 is a schematic diagram of a rounded multiplication and addition result applicable to an embodiment of the present invention, such as Fig.19 shown.

[0202] When the output format is a floating point format, the absolute value of the multiplication and addition result needs to be calculated after rounding, and the exponent needs to be corrected. The absolute value is calculated based on the sign bit of the multiplication and addition result, and the sign bit is recorded as SZ. Afterwards, the WZFF+1bit or WZFF bit of the absolute value of the multiplication and addition result after rounding is determined to be 1 based on the output format and E2, and whether E2 needs to be corrected accordingly. For example, when the output format is FP16, WZFF=11 at this time, when E2 is 0, determine whether the WZFF bit is 1, if so, E2 needs to be +1, otherwise not, when E2 is not 0, determine whether the WZFF+1bit is 1, if so, E2 needs to be +1, otherwise not, and the corrected exponent E3 is obtained.

[0203] (3) A result packaging module is used to generate the final required result based on the output format, the rounded multiplication and addition result, the corrected exponent, and the special result detection value.

[0204] When the output format is integer, first check whether E3 is greater than WZI-2. If so, directly saturate the integer result to the maximum value of the corresponding symbol. Otherwise, check whether E3 is equal to WZI-2 and whether the values ​​of the WZI and WZI-1 bits of the rounded multiplication and addition result are different. If both conditions are met, the integer result is synchronously saturated to the maximum value of the corresponding symbol. Otherwise, directly output the low WZI bits of the rounded multiplication and addition result as the output result.

[0205] When the output format is floating point, first determine whether the input special result detection value is valid. If valid, output the corresponding special result. Otherwise, determine whether the value of E3 is greater than the representation range of the normalized number in the floating point format. If it exceeds, round the result to infinity or the maximum normalized value according to the rounding mode. Otherwise, determine whether the absolute value of the multiplication and addition result is 0. If so, output 0. Otherwise, determine whether E3 is less than 0. If less than 0, normalize E3 to 0 and output the result according to the absolute value of the multiplication and addition result. Otherwise, output the result according to the exponent value and the absolute value of the multiplication and addition result.

[0206] The bit width of the module output is determined by the maximum bit width of all supported output formats. For example, when the output supports FP32, FP16, BF16, INT32, and INT16, the output bit width is 32 bits. Fig. 20 is a schematic diagram of an output result applicable to an embodiment of the present invention. If the output bit width is greater than the bit width of the output format, the redundant high bits are filled with 0, such as Fig. 20 shown.

[0207] The mixed-precision multiplication and addition hardware of the embodiment of the present invention includes: an input processing unit, a floating-point multiplier and adder, and an output processing unit, wherein: the input processing unit is used to unpack each input multiplication and addition operation factor according to a pre-configured input format, obtain the standard sign, standard exponent and standard mantissa corresponding to each multiplication and addition operation factor, and obtain the input and output special value detection result; the floating-point multiplier and adder is used to calculate the floating-point multiplication and addition operation result obtained by multiplying the operand by the scaling factor and accumulating the offset according to the standard sign, standard exponent and standard mantissa of each multiplication and addition operation factor; the output processing unit is used to normalize and round the floating-point multiplication and addition operation result according to the pre-configured output format and rounding type, obtain the target format result, and pack and output the target format result according to the input and output special value detection result. By reusing the data path of the floating-point multiplier and adder, the chip area is saved, the format conversion is integrated, the data flow is simplified, the automatic reconstruction of the multiplication and addition operation unit according to the input and output format and the requirements for quantization and dequantization of floating points and integers are realized, and the reuse rate of computing resources is improved.

[0208] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0209] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A mixed-precision multiplication-addition hardware, characterized in that: include: An input processing unit, a floating point multiplier and an output processing unit, wherein: An input processing unit, used to unpack each input multiplication and addition operation factor according to a pre-configured input format, obtain a standard sign, a standard exponent and a standard mantissa corresponding to each multiplication and addition operation factor, and obtain an input and output special value detection result; A floating-point multiplier-adder, for calculating the floating-point multiplication-addition result obtained by multiplying the operand by the scaling factor and accumulating the offset according to the standard sign, standard exponent and standard mantissa of each multiplication-addition operation factor; An output processing unit, used to normalize and round the floating-point multiplication and addition operation results according to a pre-configured output format and rounding type to obtain a target format result, and to package and output the target format result according to an input and output special value detection result; Among them, the types of multiplication and addition operation factors include: operands, scaling factors and offsets, the types of input formats and output formats include floating-point formats or integer formats with set bit widths of each multiplication and addition operation factor, and the bit width of the floating-point format includes: exponent bit width and fraction bit width.

2. The mixed-precision multiplication-addition hardware according to claim 1, characterized in that: The input processing unit includes: a mixed precision unpacking subunit and a special value detection subunit; The mixed precision unpacking subunit is used to unpack each input multiplication and addition operation factor according to the type of the multiplication and addition operation factor and the input format, and obtain the standard sign, standard exponent and standard mantissa corresponding to each multiplication and addition operation factor respectively; The special value detection subunit is used to perform special value detection on each input multiplication and addition operation factor, and perform special value detection on the output result according to the input special value detection result to obtain the output special value detection result.

3. The mixed-precision multiplication-addition hardware according to claim 2, characterized in that: The mixed precision unpacking subunit is specifically used for: If the type of the first multiplication-addition operation factor currently being unpacked is a scaling factor or an offset, parsing out a first sign, a first exponent, and a first fraction matching the first multiplication-addition operation factor; Taking the first sign as the standard sign of the factor of the first multiplication-addition operation; When the first exponent is 0 and the first fraction is not 0, 1 is used as the standard exponent of the first multiplication-addition operation factor; otherwise, the first exponent is used as the standard exponent of the first multiplication-addition operation factor; When the first exponent is 0, the integer 0 is padded before the highest digit of the first fraction, and the result of the padded 0 is used as the standard mantissa of the first multiplication and addition operation factor; otherwise, the integer 1 is padded before the highest digit of the first fraction, and the result of the padded 1 is used as the standard mantissa of the first multiplication and addition operation factor.

4. The mixed-precision multiplication-addition hardware according to claim 2, characterized in that: The mixed precision unpacking subunit is specifically used for: If the type of the second multiplication-addition operation factor currently being unpacked is an operand, when the input format of the operand is an integer format, a second sign and an integer mantissa matching the second multiplication-addition operation factor are parsed; when the input format of the operand is a floating-point format, a second sign, a second exponent, and a second fraction matching the second multiplication-addition operation factor are parsed; The second sign is used as the standard sign of the factor of the second multiplication-addition operation; If the input format for the operand is a floating point format, when the second exponent is 0 and the second fraction is not 0, 1 is used as the standard exponent of the second multiplication-addition operation factor; When the second exponent is not 0 or the second fraction is 0, the second exponent is used as the standard exponent of the second multiplication and addition operation factor; If the input format for the operand is an integer format, when the integer mantissa is 0, 0 is used as the standard exponent of the second multiplication and addition operation factor; When the integer's mantissa is not 0, Standard exponent as a factor of the second multiply-add operation; Among them, WA I is the bit width in the operand integer format, WEA F The exponent width of the operand floating point format; in WA I Greater than or equal to (WEA F +2), if the input format for the operand is the floating-point format, then when the second exponent is 0, 00 is added to the front of the highest bit of the second fraction, and the target number of 0s is added to the lowest bit of the second fraction, and the supplemented result is used as the standard mantissa of the second multiplication and addition operation factor; when the second exponent is 1, 01 is added to the front of the highest bit of the second fraction, and the target number of 0s is added to the lowest bit of the second fraction, and the supplemented result is used as the standard mantissa of the second multiplication and addition operation factor; if the input format for the operand is the integer format, the combination of the second sign and the integer mantissa is used as the standard mantissa of the second multiplication and addition operation factor; in WA I Less than (WEA F +2), if the input format for the operand is a floating-point format, then when the second exponent is 0, 00 is added to the top bit of the second fraction, and the added result is used as the standard mantissa of the second multiplication-addition operation factor; when the second exponent is 1, 01 is added to the top bit of the second fraction, and the added result is used as the standard mantissa of the second multiplication-addition operation factor; if the input format for the operand is an integer format, the second sign and the integer mantissa are combined, and the lowest bit of the combined result is filled with 0 of the target number of bits, and the added result is used as the standard mantissa of the second multiplication-addition operation factor; Among them, the target number of digits is |(WEA F +2)-WA I |, |.| are symbols for taking absolute values.

5. The mixed-precision multiplication-addition hardware according to claim 2, characterized in that: The special value detection subunit is specifically used for: When the input format of the currently detected multiplication and addition operation factor is a floating point format, if the exponent of the currently detected multiplication and addition operation factor is a maximum value and the fraction is 0, it is determined that the currently detected multiplication and addition operation factor is infinite; If the exponent of the currently detected multiplication and addition operation factor is the maximum value and the fraction is non-zero, it is determined that the currently detected multiplication and addition operation factor is a non-number; If the exponent of the currently detected multiplication-addition operation factor is 0 and the score is 0, it is determined that the currently detected multiplication-addition operation factor is 0; When the input format of the currently detected multiplication and addition operation factor is an integer format, if the integer mantissa of the currently detected multiplication and addition operation factor is 0, it is determined that the currently detected multiplication and addition operation factor is 0; When it is determined that any input multiplication and addition operation factor is a non-number, determining that the output result is a non-number; When it is determined that the input operand is 0 and the scaling factor is infinite, or the input operand is infinite and the scaling factor is 0, determining that the output result is a non-number; When it is determined that any one of the input operand and the scaling factor is infinite, the offset is infinite and the sign of the offset is opposite to the sign of the product obtained by multiplying the operand by the scaling factor, the output result is a non-number; If any of the input operands, scaling factors, and offsets is determined to be infinite, the output result is not a number or infinity.

6. The mixed-precision multiplication-addition hardware according to claim 1, characterized in that: The floating-point multiplier-adder comprises: an exponent calculation subunit, an addend order subunit, a mantissa multiplication subunit and a summation subunit; The exponent calculation subunit is used to calculate the product exponent of the operand multiplied by the scaling factor, the exponential difference between the product and the offset, the order shift value of the offset, and the multiplication and addition result exponent of the product plus the offset according to the standard exponent of each multiplication and addition operation factor; The addend alignment subunit is used to perform two's complement and exponent alignment on the standard mantissa of the offset to obtain an aligned signed addend; The mantissa multiplication subunit is used to perform signed multiplication according to the standard mantissa of the operand and the scaling factor to obtain a signed product; The summing subunit is used to sum the signed product and the signed addend to obtain a multiplication-addition result mantissa, wherein the multiplication-addition result exponent and the multiplication-addition result mantissa are combined to form a floating-point multiplication-addition operation result.

7. The mixed-precision multiplication-addition hardware according to claim 6, characterized in that: The mantissa multiplication subunit is specifically used for: Among the standard mantissas of the operand and the scaling factor, a standard mantissa with a smaller bit width is selected as a first multiplier, and another standard mantissa is selected as a second multiplier; Perform booth encoding on the first multiplier to obtain an encoding result; If the input format of the operand is a floating point format, the operand and the standard sign of the scaling factor are XORed, and when the XOR value is 1, the encoding result is inverted to obtain an updated encoding result; If the input format of the operand is an integer format, when the standard sign of the scaling factor is 1, the encoding result is inverted to obtain an updated encoding result; A partial product is generated according to the current encoding result and the second multiplier, and the partial product is compressed using a tree adder based on a carry-save adder to obtain the signed product.

8. The mixed-precision multiplication-addition hardware according to claim 1, characterized in that: The output processing unit includes: a multi-precision normalization subunit, a rounding subunit and a result packing subunit; The multi-precision normalization subunit is used to determine a shift value during normalization according to a pre-configured output format, and perform normalization processing on the floating-point multiplication and addition operation result according to the shift value to obtain a normalized operation result; The rounding subunit is used to determine the carry value of the normalized operation result according to the pre-configured rounding type, and calculate the rounded multiplication-addition result mantissa and the corrected multiplication-addition result exponent according to the carry value; The result packing subunit is used to generate a target format result according to the output format, the rounded multiplication and addition result mantissa, the corrected multiplication and addition result exponent, and the input and output special value detection result, and to pack and output the target format result.

9. The mixed-precision multiplication-addition hardware according to claim 8, characterized in that: The multi-precision normalization subunit is specifically used for: Perform redundant sign bit detection on the mantissa of the multiplication and addition result to obtain the redundant sign bit number, and determine the sign bit padding number according to the maximum bit width of all integer formats and the maximum fractional bit width of all floating point formats that can be supported by the output format; Supplement the high bit of the mantissa of the multiplication-addition result with a sign bit that matches the number of sign bit padding to obtain the mantissa of the multiplication-addition result after padding; Determine the number of decimal point left shifts after normalization according to the output format, and modify the multiplication-addition result exponent to the corrected multiplication-addition result exponent according to the number of decimal point left shifts; Calculate the number of left shifts corresponding to the mantissa of the multiplication-addition result according to the number of redundant sign bits, the multiplication-addition result exponent, and the output format; According to the number of left shifts, the mantissa of the padded multiplication-addition result is left-shifted and normalized to obtain a normalized multiplication-addition result mantissa, and the current corrected multiplication-addition result exponent is modified to a new corrected multiplication-addition result exponent that matches the number of left shifts.

10. The mixed-precision multiplication-addition hardware according to claim 8, characterized in that: The rounding subunit is specifically used for: After the integer 0 is added before the highest bit of the mantissa of the normalized multiplication and addition result, the valid value part and the discarded value part are identified in the mantissa of the normalized multiplication and addition result after the 0 is added; Generate a carry value according to the rounding mode and the discarded value part, and calculate the sum of the effective value part and the carry value as the mantissa of the multiplication and addition result after rounding; If the output format is a floating point format, the absolute value of the rounded multiplication-addition result mantissa is calculated as the new rounded multiplication-addition result mantissa, and the current corrected multiplication-addition result exponent is modified again according to the fractional bit width of the new rounded multiplication-addition result mantissa to obtain a new corrected multiplication-addition result exponent.

11. The mixed-precision multiplication-addition hardware according to claim 8, characterized in that: The result packaging subunit is specifically used for: If the output format is an integer format, when the corrected multiplication-addition result exponent is greater than the bit width of the integer format -2, or when the corrected multiplication-addition result exponent is equal to the bit width of the integer format -2 and the bit width of the integer format in the rounded multiplication-addition result mantissa is different from the value of the bit width of the integer format -1, the rounded multiplication-addition result mantissa is saturated to the maximum value of the corresponding sign as the output result; Otherwise, the low-order data matching the bit width of the integer format is intercepted from the mantissa of the multiplication-addition result after rounding as the output result; If the output format is a floating point format, then when the input and output special value detection result is valid, the matching output special value detection result is output; otherwise, it is determined whether the corrected multiplication and addition result exponent exceeds the representation range of the normalized number in the floating point format; If exceeded, the result is rounded to infinity or the maximum normalized value, depending on the rounding mode; If it does not exceed, determine whether the mantissa of the multiplication and addition result after rounding is 0; If it is 0, then output 0; if it is not 0, then when the corrected multiplication-addition result exponent is less than 0, the corrected multiplication-addition result exponent is normalized to 0 and then the matching output result is output; When the corrected multiplication-addition result exponent is greater than or equal to 0, the matching output result is directly output.

12. The multiplication and addition hardware according to any one of claims 1 to 11, characterized in that: The multiplication and addition hardware can perform on-chip reconstruction of configuration information in real time according to configuration information of input format, output format and rounding type matching the current computing scenario to meet the computing requirements of the current computing scenario.

Images