A multiple precision dot product operation hardware

Abstract

The application discloses a kind of multi-precision dot product operation hardware, comprising: input processing unit is according to input data format, unpacking and obtaining symbol, index and mantissa distribution pattern;Determine the special value of dot product result, calculate product symbol;Dot product operator, according to each index distribution pattern, calculate dot product index and product shift value;According to symbol, mantissa distribution pattern and product symbol, calculate product or partial product, according to input data format and / or shift value, carry out shift alignment and summation, sum dot product operation intermediate result obtains dot product result;Output processing unit, according to standardization output format, rounding type and the index of dot product result, round and normalize dot product mantissa, according to the special value of dot product result, pack output specification processing result.The hardware supports multiple precision input data format, expands the application scene of hardware, by multiplexing dot product operator, reduce the area and power consumption of design, improve hardware resource utilization and calculation efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of integrated chip, and particularly relates to a multi-precision dot product operation hardware. BACKGROUND

[0002] With the rapid development of artificial intelligence large models, the computing power demand of matrix operation increases sharply, and matrix operation hardware becomes a key way to provide the required computing power. In order to meet the diversified business needs in artificial intelligence tasks, the basic hardware of matrix operation, especially the multiply-add operation, needs to support different precision data formats to adapt to different application scenarios.

[0003] In the prior art, the multiply-add unit supporting multi-precision data format is realized by low-precision combination strategy to realize efficient utilization of the basic multiplier. However, these technical solutions mainly focus on floating-point data format operation and fail to cover integer data format and low-precision floating-point data format such as FP8 (float point 8, 8-bit floating-point data format), which limits their application range and restricts the computing power in scenarios requiring low-precision data format, increases the complexity of calculation, causes waste of computing power resources, reduces resource utilization and calculation efficiency. SUMMARY

[0004] The present application provides a multi-precision dot product operation hardware to solve the problems of few data format precisions supported by the dot product operation hardware, few applicable application scenarios, low hardware resource utilization and low calculation efficiency.

[0005] According to an aspect of an embodiment of the present application, a multi-precision dot product operation hardware is provided, comprising: an input processing unit, a dot product operation unit and an output processing unit, wherein:

[0006] The input processing unit is configured to perform unpacking processing on the two input operation arrays according to a pre-configured input data format, and obtain a sign distribution pattern, an exponent distribution pattern and a mantissa distribution pattern corresponding to each operation array according to the unpacking result and the input data format; obtain a special value judgment result of the dot product result, and calculate the sign of the product of each pair of operation numbers in the two operation arrays.

[0007] The dot product operator is used to calculate the exponent of the dot product result and the shift value of the product of each pair of operation numbers according to the index distribution pattern of each operation array, and to calculate each product or partial product between each pair of operation numbers in the two operation arrays according to the sign distribution pattern, the mantissa distribution pattern of each operation array and the sign of the product of each pair of operation numbers; the multiple dot product operation intermediate results are obtained after at least one shift alignment and at least one summation processing of each product or partial product according to the input data format and / or the shift value of the product of each pair of operation numbers; the dot product result is obtained by performing summation operation on the multiple dot product operation intermediate results; the dot product result includes the dot product mantissa and the dot product sign;

[0008] The output processing unit is used to perform rounding and normalization processing on the dot product mantissa according to the pre-configured standardized output format, the rounding type and the exponent of the dot product result; the normalized dot product mantissa, the normalized dot product sign and the normalized dot product exponent satisfying the standardized output format are obtained as the normalization processing result; and the normalization processing result is packaged and output according to the special value judgment result of the dot product result.

[0009] The multiple-precision dot product operation hardware of the embodiment includes: an input processing unit used to unpack and obtain the sign, the index distribution pattern and the mantissa distribution pattern according to the input data format; to judge the special value of the dot product result and to calculate the product sign; a dot product operator used to calculate the dot product exponent and the product shift value according to each index distribution pattern; to calculate the product or partial product according to the sign, the mantissa distribution pattern and the product sign; to perform shift alignment and summation according to the input data format and / or the shift value; to obtain the dot product result by summing the dot product operation intermediate results; and an output processing unit used to round and normalize the dot product mantissa according to the standardized output format, the rounding type and the exponent of the dot product result; and to package and output the normalization processing result according to the special value of the dot product result. The hardware supports multiple-precision input data formats, expands the application scenarios of the hardware, helps to reduce the area and power consumption of the design by reusing the calculation unit, and improves the hardware resource utilization and the calculation efficiency.

[0010] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creating laborious work.

[0012] Figure 1 is a structural diagram of a multi-precision dot product operation hardware according to an embodiment of the present application;

[0013] Figure 2 is a structural diagram of another multi-precision dot product operation hardware according to another embodiment of the present application;

[0014] Figure 3 is a structural diagram of still another multi-precision dot product operation hardware according to still another embodiment of the present application;

[0015] Figure 4 is a reconfigurable multi-precision dot product operation unit according to an embodiment of the present application;

[0016] Figure 5 is an index and arrangement of data in an input operation array according to an embodiment of the present application;

[0017] Figure 6 is data required for unpacking of different data formats according to an embodiment of the present application;

[0018] Figure 7 is an optimal arrangement of exponents unpacked from different data formats according to an embodiment of the present application;

[0019] Figure 8 is a principle of arrangement of signs of floating point formats according to an embodiment of the present application;

[0020] Figure 9 is an optimal arrangement of mantissas unpacked from different data formats according to an embodiment of the present application;

[0021] Figure 10 is a principle of arrangement of detection results of special values of floating point formats according to an embodiment of the present application;

[0022] Figure 11 is a circuit of a preferred sign processing and special result judging module according to an embodiment of the present application;

[0023] Figure 12 is an implementation of a preferred exponent processing and shift value calculating module according to an embodiment of the present application;

[0024] Figure 13 is a preferred implementation of a multi-precision dot product module according to an embodiment of the present application;

[0025] Figure 14 is a type of mantissa multiplier included in a preferred implementation according to an embodiment of the present application;

[0026] Figure 15 is a preferred segmenting scheme for FP64 data format used in embodiments of the present application;

[0027] Figure 16 is a preferred exponent multiplier distribution scheme for FP64 data format used in embodiments of the present application;

[0028] Figure 17 is a preferred segmenting scheme for FP32 data format used in embodiments of the present application;

[0029] Figure 18 is a preferred exponent multiplier distribution scheme for FP32 data format used in embodiments of the present application;

[0030] Figure 19 is a preferred exponent multiplier distribution scheme for TF32 data format used in embodiments of the present application;

[0031] Figure 20 is a preferred exponent multiplier distribution scheme for BF16 or FP16 data format used in embodiments of the present application. DETAILED DESCRIPTION

[0032] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the person skilled in the art without creative labor should belong to the scope of protection of the present application.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0034] Embodiment one

[0035] Figure 1 A structural diagram of a multi-precision dot product operation hardware is provided for an embodiment of the present application. The embodiment can be applied to dot product operation on multi-precision data format data, especially in the case of multi-precision data dot product operation in high-performance computing and artificial intelligence application scenarios. As shown in the figure, the multi-precision dot product operation hardware includes an input processing unit 110, a dot product operator 120, and an output processing unit 130, wherein: Figure 1

[0036] The input processing unit 110 is configured to unpack the two input operation arrays according to a pre-configured input data format, and obtain a sign distribution pattern, an exponent distribution pattern, and a mantissa distribution pattern corresponding to each operation array according to the unpacking result and the input data format. The input processing unit 110 is also configured to obtain a special value judgment result of the dot product result, and calculate the sign of the product of each pair of operation numbers in the two operation arrays.

[0037] The dot product operator 120 is configured to calculate the exponent of the dot product result and the shift value of the product of each pair of operation numbers in the two operation arrays according to the exponent distribution pattern of each operation array. The dot product operator 120 is also configured to calculate each product or partial product between each pair of operation numbers in the two operation arrays according to the sign distribution pattern, the mantissa distribution pattern of each operation array, and the sign of the product of each pair of operation numbers. The dot product operator 120 is further configured to obtain a plurality of dot product operation intermediate results by performing at least one shift alignment and at least one summation processing on each product or partial product according to the input data format and / or the shift value of the product of each pair of operation numbers. The dot product operator 120 is also configured to obtain the dot product result by performing summation operation on the plurality of dot product operation intermediate results. The dot product result includes a dot product mantissa and a dot product sign.

[0038] The output processing unit 130 is configured to round and normalize the dot product mantissa according to a pre-configured standardized output format, a rounding type, and the exponent of the dot product result. The output processing unit 130 is also configured to obtain a normalized dot product mantissa, a normalized dot product sign, and a normalized dot product exponent that satisfy the standardized output format as a normalization processing result, and to pack and output the normalization processing result according to the special value judgment result of the dot product result.

[0039] ​In the embodiment of the present application, the pre-configured input data format can be understood as the input data format supported by the multi-precision dot product operation hardware provided by the embodiment of the present application, and can specifically include: FP64 (double-precision floating-point number, 64 bits), FP32 (single-precision floating-point number, 32 bits), TF32 (Tensor Float 32, a floating-point number format used for deep learning, with a precision similar to FP16 while maintaining the dynamic range of FP32), FP16 (half-precision floating-point number, 16 bits), BF16 (Brain Floating Point 16, 16-bit floating-point number used for deep learning), FP8 (8-bit floating-point number), and INT8 (8-bit integer) format. Among them, FP8 includes two formats: FP8_E4M3 (with 4-bit exponent and 3-bit mantissa, providing a smaller numerical range and higher precision), hereinafter referred to as E4M3; and FP8_E5M2 (with 5-bit exponent and 2-bit mantissa, providing a larger numerical range and smaller precision), hereinafter referred to as E5M2.

[0040] The multi-precision dot product operation hardware provided by the embodiment of the present application supports high-precision data formats including FP64 and FP32, is suitable for supercomputing, also supports low-precision data formats such as BF16 and FP8, meets the needs of artificial intelligence, and is suitable for application scenarios of supercomputing and artificial intelligence integration. In addition, the hardware allows dynamic adjustment of the range of supported data formats according to specific application requirements, such as expanding to support more data formats, or simplifying the supported data formats to optimize the performance of specific workloads.

[0041] Correspondingly, the operation modes supported by the multi-precision dot product operation hardware provided by the embodiment of the present application can specifically include: FP64·FP64, FP32·FP32, TF32·TF32, FP16·FP16, BF16·BF16, E4M3·E4M3, E5M2·E5M2, E4M3·E5M2, E5M2·E4M3, and INT8·INT8. Among them, · represents dot product operation, that is, The data format before the dot product operation represents the format of the input operation array A, and the data format after the dot product operation represents the format of another input operation array B. The dot product is the result of multiplying the corresponding elements of two vectors and then summing them up, and the sign of the product is determined by the signs of the two operands, that is, the product of two positive numbers is positive, the product of two negative numbers is also positive, and the product of a positive number and a negative number or the product of a negative number and a positive number is negative.

[0042] The floating point number is composed of a sign bit, an exponent bit and a mantissa bit. The sign bit is usually the leftmost bit, which is used to represent the positive or negative of the value. The sign distribution pattern can be understood as the distribution of the sign bits of all floating point numbers in the input array, which reflects the distribution of positive and negative numbers in the input array. The exponent bit is used to represent the range of the value. In the representation of floating point numbers, the exponent bit determines the magnitude of the value. The exponent distribution pattern can be understood as the distribution of the exponent bits of all floating point numbers in the input array, which reflects the range and sparsity of the data in the input array. The mantissa bit is used to represent the precision of the value. In the representation of floating point numbers, the mantissa bit determines the precision of the value. The mantissa distribution pattern can be understood as the distribution of the mantissa bits of all floating point numbers in the input array, which reflects the precision of the data in the input array.

[0043] The input processing unit 110 performs unpacking processing on the floating point numbers or integers in each of the two input operation arrays according to the pre-configured input data format, to obtain the sign distribution pattern, the exponent distribution pattern and the mantissa distribution pattern corresponding to the data format. The two operation arrays have the same data bit width, and one preferred value can be 512 bits. For floating point numbers, the sign distribution pattern shows the distribution of positive and negative values, the exponent distribution pattern shows the range and sparsity of the values, and the mantissa distribution pattern shows the precision of the values. For integers, since there is no exponent in the format and the values are already signed, the sign distribution and the exponent distribution are both set to 0, and the mantissa distribution pattern shows the distribution of the integer values.

[0044] The input processing unit 110 detects special values (zero, infinity and NaN) in the floating point numbers in the input operation arrays to specially process the corresponding floating point operations, for example, the result of multiplying any number by zero is zero, the result of multiplying any number by infinity is infinity, and the result of operations involving NaN is usually NaN. The special value judgment result of the dot product result is obtained, and the sign of the product of each pair of operation numbers in the two operation arrays is calculated and delivered to the dot product operator 120.

[0045] In floating point multiplication, the exponent of the product of two floating point numbers is the sum of the exponents of the two numbers. In addition, in order to align the decimal point, the mantissa needs to be shifted according to the difference of the exponents. The shift value of the mantissa can be determined by calculating the difference of the exponents of each pair of operation numbers in the two operation arrays.

[0046] In the dot product operation, the result of multiplying each pair of operands (i.e., corresponding elements in the two vectors) is called a product. In some data format configurations (e.g., FP32 and FP64), the full product cannot be obtained using a multiplier for the purpose of hardware multiplexing of the multiplier, and the result output by the multiplier is then called a partial product, indicating that it is a part of the product. After obtaining the partial product, a shift and addition operation need to be applied to multiple partial products to calculate the product. The final dot product result is the sum of all partial products or products.

[0047] The dot product operator 120 calculates each product or partial product between each pair of operands in the two operand arrays. Each product or partial product needs to be aligned with other partial products to ensure that the product or partial product has the same exponent, and then added. Therefore, when performing large-scale calculations or processing different precision data, one or more shift alignment and summation processes need to be performed to ensure that all products or partial products are correctly accumulated to obtain multiple dot product operation intermediate results. Finally, the summation operation is performed on the multiple dot product operation intermediate results to obtain the dot product result.

[0048] In addition, the dot product operator 120 can be multiplexed under different data formats, that is, the same dot product operator can process different types of data, which not only reduces the demand for dedicated hardware resources, thereby reducing the design complexity and area occupation of the integrated circuit, but also reduces power consumption, improves energy efficiency, and improves hardware resource utilization.

[0049] The standardized output format can be a limitation on the output format of the calculation result, which is generally a limitation on the mantissa and can also include a limitation on the sign and exponent. Preferably, the FP64 format is selected to output the dot product result to ensure high precision and accuracy of the calculation and effectively reduce the cumulative error that may occur in the data processing process. In addition, the standardized output format can be converted into multiple data formats including FP64 according to specific application requirements, which not only ensures the accuracy of the hardware output result, but also improves the usability of the data and the adaptability of the system.

[0050] The rounding type refers to the method of processing values that exceed the representation range in floating-point number operations, which can include rounding, rounding to zero, rounding to positive infinity, and rounding to negative infinity. Through the support of various rounding modes, the multi-precision dot product operation hardware proposed in the embodiments of the present application can flexibly select appropriate rounding strategies according to different application scenarios and precision requirements. For example, the rounding mode is selected to reduce the cumulative error while maintaining the balance of the value, or the rounding to zero mode is selected to avoid systematic deviation of the value. In addition, the rounding mode supported by the hardware can be extended or reduced according to actual use requirements, further improving the adaptability and flexibility of the system.

[0051] In one specific example, the FP64 format is selected for outputting the dot product result, and if the output result is not a special value, the output processing unit 130 normalizes the mantissa of the dot product result according to the pre-configured normalized output format, the rounding type, and the exponent of the dot product result, to eliminate the redundant sign bits in the mantissa of the dot product result when converted to the normalized output format. Then, the mantissa of the floating point number is rounded according to the rounding type. In particular, a certain number of shift-out bits need to be reserved when the mantissa is right-shifted, which are used for the rounding process after normalization. Finally, according to the special value judgment result of the dot product result, if the dot product result is zero, the zero value is directly output; if the dot product result is infinity or NaN, the corresponding special value is output; if the output result is not a special value, the normalized dot product mantissa, the normalized dot product sign, and the normalized dot product exponent after normalization and rounding processing are combined to form the normalization processing result, and these parts are packaged into a binary string, or converted into hexadecimal or other formats as needed and output.

[0052] The multi-precision dot product operation hardware of the embodiments of the present application includes: an input processing unit that unpacks and obtains the sign, exponent, and mantissa distribution pattern according to the input data format; a special value judgment unit that judges the special value of the dot product result and calculates the product sign; a dot product operation unit that calculates the dot product exponent and the product shift value according to the distribution pattern of each exponent, calculates the product or partial product according to the sign, mantissa distribution pattern, and product sign, and performs shift alignment and summation according to the input data format and / or shift value, and obtains the dot product result by summing the intermediate results of the dot product operation; and an output processing unit that rounds and normalizes the mantissa of the dot product according to the normalized output format, rounding type, and exponent of the dot product result, and packs and outputs the normalization processing result according to the special value of the dot product result. The hardware supports multiple precision input data formats, expands the application scenarios of the hardware, helps to reduce the area and power consumption of the design by reusing the calculation unit, and improves the hardware resource utilization and calculation efficiency.

[0053] Optionally, on the basis of the above embodiments, the output processing unit 130 can specifically include: a normalization and rounding unit, and a result packing unit, wherein:

[0054] The normalization and rounding unit is configured to detect the number of sign bits contained in the input dot product result, and perform primary normalization on the dot product result according to the number of sign bits and the exponent of the input dot product result; perform rounding processing on the dot product result after the primary normalization according to the rounding type to obtain a rounding result; take the absolute value of the rounding result and perform secondary normalization to obtain the normalized dot product mantissa, normalized dot product sign, and normalized dot product exponent that satisfy the normalized output format as the normalization processing result;

[0055] The result packaging unit is configured to, when the special value judgment result of the dot product result indicates that the dot product result is a special value, package and output the special value; and when the special value judgment result of the dot product result indicates that the dot product result is not a special value, package the normalized dot product mantissa, the normalized dot product sign and the normalized dot product exponent to obtain a packaging result that satisfies the data representation range of the standardized output format and output the packaging result.

[0056] In one specific example, the FP64 standardized output format is selected to output the dot product result. The normalization and rounding unit is configured to normalize and round the input dot product result to obtain a dot product mantissa result and a sign that satisfy the requirements of the FP64 format. The normalization and rounding unit first detects the number of sign bits of the input dot product result, and performs initial normalization on the dot product result in combination with the exponent value. During the normalization process, the sign bits remain unchanged because the sign bits only indicate the positive or negative of the value and are not related to the adjustment of the size or precision. The exponent value indicates the order of magnitude of the dot product result. During the normalization process, it is necessary to ensure that the exponent value is within the effective range, i.e., the exponent value cannot exceed the maximum or minimum exponent value of the floating-point number representation. If the exponent value exceeds the range, it needs to be adjusted. For example, if the exponent value exceeds the maximum exponent value, it indicates that the data exceeds the maximum range that can be represented by the standardized output format, and at this time, it is necessary to decide according to the rounding type whether to set the output to be infinite or the maximum value that can be represented by the standardized output format. The mantissa represents the significant digit part of the dot product result. During the normalization process, when the exponent value is greater than 0, it is necessary to ensure that the highest bit of the mantissa is 1. When the highest bit of the mantissa is less than 1, the mantissa needs to be left shifted until the highest bit is 1. Each left shift by one bit decreases the exponent value by 1. This is because left shifting the mantissa is equivalent to multiplying the value by 2, and decreasing the exponent value by 1 is equivalent to dividing the value by 2, thereby maintaining the invariance of the value. When the highest bit of the mantissa is greater than 1, the mantissa needs to be right shifted until the highest bit is 1. Each right shift by one bit increases the exponent value by 1. This is because right shifting the mantissa is equivalent to dividing the value by 2, and increasing the exponent value by 1 is equivalent to multiplying the value by 2, thereby maintaining the invariance of the value. During the normalization process, when the exponent value is less than or equal to 0, the mantissa needs to be right shifted and the value of the exponent needs to be increased accordingly until the exponent value becomes 1. In this way, a non-normalized value is generated. By normalizing the dot product result into a standard form, it is ensured that the format requirements of the IEEE 754 standard floating-point number are met.

[0057] After the normalized result is rounded, the preferred and supported rounding modes include rounding to 0, rounding to positive infinity, rounding to negative infinity, and rounding to the nearest even number. After rounding, the absolute value of the rounded result is taken, and the result is normalized again to obtain the normalized dot product mantissa, the normalized dot product sign, and the normalized dot product exponent, which are combined to obtain the normalized processing result. The floating-point number is adjusted to the required precision by rounding processing through the rounding mode, and the mantissa is converted to an unsigned number required by the FP64 format by taking the absolute value. Finally, the result after taking the absolute value is adjusted back to the standard form by normalizing again, thereby obtaining the standard mantissa.

[0058] The result packaging unit packages the result according to the input special result, sign, exponent, and dot product mantissa to form the dot product result in the FP64 format. When the special result is detected as infinite or non-number in the sign processing and special result judgment module, the corresponding special result is packaged and output. For example, in the floating-point number representation, infinity is usually represented by all 1 exponent bits and all 0 mantissa bits, and non-number (NaN) is usually represented by all 1 exponent bits and non-all 0 mantissa bits. The output of these special values ensures the accuracy and consistency of the calculation result.

[0059] If no special result is detected, the normalized dot product mantissa, the normalized dot product sign, and the normalized dot product exponent are packaged. In the packaging process, it is determined whether the exponent exceeds the representation range of the FP64 format. If the number exceeds the representation range, it is rounded to infinity or the maximum value according to the rounding mode and packaged and output. If the number does not exceed the representation range, the result that meets the standardized output format is packaged and output, ensuring that the dot product result is quickly and accurately output when it is a special value, avoiding incorrect data processing and propagation, and improving the reliability of hardware calculation. At the same time, for non-special values, the normalized dot product sign, the normalized dot product exponent, and the normalized dot product mantissa are directly spliced into the output FP64 format data, realizing fast data packaging operation.

[0060] Optionally, on the basis of the above embodiments, the multi-precision dot product operation hardware can be configured according to the configuration information of the input data format, the standardized output format, and the rounding type matched with the current calculation scenario, to realize on-chip reconstruction of the configuration information in real time, so as to meet the calculation requirement of the current calculation scenario.

[0061] The multi-precision dot product operation hardware provided in the embodiments of the present application can change the working mode of the hardware in real time according to the actual task requirement, without the need for corresponding static configuration for each possible combination of data format and calculation requirement.

[0062] In a specific example, the multi-precision dot product operation hardware can automatically identify the format of the input data and automatically adjust the working mode according to the format. For example, if the input data is in FP16 format, the hardware will dynamically reconfigure on-chip configuration, such as changing the bit width, precision, data path or other related parameters of the operation unit, to run in the most suitable way for processing FP16 data and generate the calculation result. The multi-precision dot product operation hardware can dynamically configure and reconfigure on-chip in real time according to the specific needs of the current calculation scene, such as input data format, output format and rounding type, to adapt to different calculation tasks, improve the flexibility and adaptability of the hardware, and realize high efficiency performance while meeting the diversified calculation needs, whether it is high-precision scientific calculation or deep learning application focusing on efficiency, to improve hardware resource utilization and calculation efficiency.

[0063] Optionally, on the basis of each of the above embodiments, the multi-precision dot product operation hardware can insert any number of pipeline registers.

[0064] Pipeline registers refer to a component for temporarily storing intermediate results. In order to ensure data correlation, all data paths from the original input to the original output must pass through the same number of pipeline registers to ensure synchronization and correct transmission of data between different stages. By inserting any number of pipeline registers as appropriate, complex combinational logic can be divided into shorter paths, reducing the number of logic stages, shortening the data path, and thus improving the throughput of the hardware, optimizing the operation delay, and improving the system working frequency, thereby improving the hardware resource utilization and calculation efficiency.

[0065] Embodiment two

[0066] Figure 2 Another multi-precision dot product operation hardware structure diagram is provided for embodiment two of the present application. This embodiment is a refinement of the input processing unit in the above-mentioned embodiments, and specifically includes an input unpacking subunit, a special value detection unit, a sign processing unit and a special result judgment unit.

[0067] Correspondingly, as shown in Figure 2 The multi-precision dot product operation hardware includes an input unpacking subunit 210, a special value detection unit 220, a sign processing unit 230, a special result judgment unit 240, a dot product operator 250 and an output processing unit 260, wherein:

[0068] The input unpacking subunit 210 is configured to determine the unpacking quantity and the unpacking position corresponding to each operation array according to the input data format after obtaining a plurality of data packets corresponding to each operation array; split the sign, the exponent and the mantissa of each operation number in each operation array from the data packets according to the unpacking quantity and the unpacking position; and fill the split sign, the exponent and the mantissa into a matching style template according to a data arrangement mode matching the input data format, to obtain a sign distribution pattern, an exponent distribution pattern and a mantissa distribution pattern corresponding to each operation array.

[0069] The special value detection unit 220 is configured to determine the zero value detection result, the infinite value detection result and the non-number detection result of each operation number in each operation array according to the input data format and the exponent and the mantissa of each operation number in each operation array; and fill the zero value detection result, the infinite value detection result and the non-number detection result into a matching style template according to a data arrangement mode matching the input data format, to obtain a zero value distribution pattern, an infinite value distribution pattern and a non-number distribution pattern corresponding to each operation array.

[0070] The sign processing unit 230 is configured to perform a bitwise XOR operation on the sign distribution pattern corresponding to each operation array, to obtain the sign of the product of each pair of operation numbers in the two operation arrays.

[0071] The special result judgment unit 240 is configured to perform a logical judgment on the zero value distribution pattern, the infinite value distribution pattern and the non-number distribution pattern of each operation array through a pre-connected logic gate array, to obtain the zero value judgment result, the infinite value judgment result and the non-number judgment result corresponding to the dot product result.

[0072] The dot product operator 250 is configured to calculate the exponent of the dot product result and the shift value of the product of each pair of operation numbers in the two operation arrays according to the exponent distribution pattern of each operation array; calculate each product or partial product between each pair of operation numbers in the two operation arrays according to the sign distribution pattern, the mantissa distribution pattern of each operation array and the sign of the product of each pair of operation numbers; perform at least one shift alignment and at least one summation processing on each product or partial product according to the input data format and / or the shift value of the product of each pair of operation numbers, to obtain a plurality of dot product operation intermediate results; and perform a summation operation on the plurality of dot product operation intermediate results, to obtain the dot product result; the dot product result includes a dot product mantissa and a dot product sign.

[0073] The output processing unit 260 is configured to round and normalize the dot product mantissa according to the pre-configured standardized output format, rounding type and the exponent of the dot product result, to obtain the normalized dot product mantissa, the normalized dot product sign and the normalized dot product exponent as the result of the normalization processing, and to pack and output the result of the normalization processing according to the special value judgment result of the dot product result.

[0074] The input operation arrays have the same data bit width, and the preferred value is 512 bits. Specifically, the input unpacking subunit 210 determines the unpacking quantity and unpacking position corresponding to each operation array according to the input data format after obtaining a plurality of data packets corresponding to each operation array. When the data format is FP64, FP32, TF32, FP16, BF16, FP8 and INT8 format, respectively, 2, 8, 16, 32, 32, 64 and 64 floating point numbers or integers in each input operation array need to be unpacked. When the data format is FP64 or FP32, only a part of the floating point numbers in each input operation array need to be unpacked. When the data format is FP64, each input operation array has 8 floating point FP64 format numbers, but only 2 floating point FP64 format numbers in each input operation array need to be unpacked. The indexes of the two floating point numbers can be any two different indexes in 1-8, but the indexes used for each input operation array are the same. A preferred way is to use the floating point numbers with indexes 1 and 5. When the data format is FP32, each input operation array has 16 floating point FP32 format numbers, but only 8 floating point FP32 format numbers in each input operation array need to be unpacked. The indexes of the two floating point numbers can be any 8 different indexes in 1-16, but the indexes used for each input operation array are the same. A preferred way is to use the floating point numbers with indexes 1, 2, 5, 6, 9, 10, 13 and 14. When the data format is the remaining format, all floating point numbers in each input operation array need to be unpacked.

[0075] In a specific example, when the total bit width of the split exponent is 384 bits, the total bit width of the split sign is 64 bits, and the total bit width of the split mantissa is 512 bits, after the sign, exponent and mantissa corresponding to each operation number in each operation array are split, the corresponding sign distribution pattern, exponent distribution pattern and mantissa distribution pattern are obtained according to the data arrangement mode matching the input data format.

[0076] The exponential distribution pattern can be constructed in the following manner: when the data format is floating point format, the exponent field is extracted from the floating point format according to the IEEE 754 standard; when the data format is FP64, 2 FP64 data need to be split, and the exponent of each FP64 data is 11 bits, which is located in the lower 11 bits of the high and low 192 bits, and the values of the remaining bit fields are 0.

[0077] When the data format is FP32, 8 FP32 data need to be split, and the exponent of each FP32 data is 8 bits, which is located in the lower 8 bits of the 8 48-bit parts of the 384-bit part, and the values of the remaining bit fields are 0.

[0078] When the data format is TF32, 16 TF32 data need to be split, and the exponent of each TF32 data is 8 bits, which is located in the lower 8 bits of the 16 24-bit parts of the 384-bit part, and the values of the remaining bit fields are 0.

[0079] When the data format is FP16, 32 FP16 data need to be split, and the exponent of each FP16 data is 5 bits, which is located in the lower 5 bits of the 32 12-bit parts of the 384-bit part, and the values of the remaining bit fields are 0.

[0080] When the data format is BF16, similar to FP16, 32 BF16 data need to be split, and the exponent of each BF16 data is 8 bits, which is located in the lower 8 bits of the 32 12-bit parts of the 384-bit part, and the values of the remaining bit fields are 0.

[0081] When the data format is E5M2 (FP8_E5M2), 64 FP8 data need to be split, and the exponent of each FP8 data is 5 bits, which is located in the lower 5 bits of the 64 6-bit parts of the 384-bit part, and the values of the remaining bit fields are 0.

[0082] When the data format is E4M3 (FP8_E4M3), 64 FP8 data also need to be split, and the exponent of each FP8 data is 4 bits, which is located in the lower 4 bits of the 64 6-bit parts of the 384-bit part, and the values of the remaining bit fields are 0.

[0083] For data format INT8, the exponent value is all 0.

[0084] The symbol distribution pattern can be constructed in the following manner: when the data format is floating point format, the sign bit field is extracted from the floating point format according to the IEEE 754 standard; when the data format is FP64, 2 FP64 data need to be split, and each FP64 data is split into 1 bit, which is located at the lowest 1 bit of the high and low 32 bits, and the values of the remaining bit fields are 0.

[0085] When the data format is FP32, 8 FP32 data need to be split, and each FP32 data is split into 1 bit, which is located at the low 1 bit of the 8 8-bit bits divided from the 64-bit bits, and the values of the remaining bit fields are 0.

[0086] When the data format is TF32, 16 TF32 data need to be split, and each TF32 data is split into 1 bit, which is located at the low 1 bit of the 16 4-bit bits divided from the 64-bit bits, and the values of the remaining bit fields are 0.

[0087] When the data format is FP16, 32 FP16 data need to be split, and each FP16 data is split into 1 bit, which is located at the low 1 bit of the 32 2-bit bits divided from the 64-bit bits, and the values of the remaining bit fields are 0.

[0088] When the data format is BF16, 32 BF16 data need to be split, and each BF16 data is split into 1 bit, which is located at the low 1 bit of the 32 2-bit bits divided from the 64-bit bits, and the values of the remaining bit fields are 0.

[0089] When the data format is E5M2, 64 E5M2 data need to be split, and each E5M2 data is split into 1 bit, which is located at the 64 1-bit bits divided from the 64-bit bits.

[0090] When the data format is E4M3, 64 E4M3 data need to be split, and each E4M3 data is split into 1 bit, which is located at the 64 1-bit bits divided from the 64-bit bits.

[0091] For data format INT8, the symbol value is all 0.

[0092] The tail number distribution pattern can be constructed in the following manner: when the data format is floating point format, the tail number bit field is extracted from the floating point format according to the IEEE 754 standard; when the data format is FP64, 2 FP64 data need to be split and processed, and the tail number obtained by processing each FP64 data is 53 bits, which is located in the lowest 53 bits of the high and low 256 bits, and the values of the remaining bit fields are 0.

[0093] When the data format is FP32, 8 FP32 data need to be split and processed, and the tail number obtained by processing each FP32 data is 24 bits, which is located in the low 24 bits of the 8 64-bit bits evenly divided from 512 bits, and the values of the remaining bit fields are 0.

[0094] When the data format is TF32, 16 TF32 data need to be split and processed, and the tail number obtained by processing each TF32 data is 11 bits, which is located in the low 11 bits of the 16 32-bit bits evenly divided from 512 bits, and the values of the remaining bit fields are 0.

[0095] When the data format is FP16, 32 FP16 data need to be split and processed, and the tail number obtained by processing each FP16 data is 11 bits, which is located in the low 11 bits of the 32 16-bit bits evenly divided from 512 bits, and the values of the remaining bit fields are 0.

[0096] When the data format is BF16, 32 BF16 data need to be split and processed, and the tail number obtained by processing each BF16 data is 8 bits, which is located in the low 8 bits of the 32 16-bit bits evenly divided from 512 bits, and the values of the remaining bit fields are 0.

[0097] When the data format is E5M2, 64 E5M2 data need to be split and processed, and the tail number obtained by processing each E5M2 data is 3 bits, which is located in the low 3 bits of the 64 8-bit bits evenly divided from 512 bits, and the values of the remaining bit fields are 0.

[0098] When the data format is E4M3, 64 E4M3 data need to be split and processed, and the tail number obtained by processing each E4M3 data is 4 bits, which is located in the low 4 bits of the 64 8-bit bits evenly divided from 512 bits, and the values of the remaining bit fields are 0.

[0099] For data format INT8, the tail number value is the close arrangement of 64 INT8 data.

[0100] The special value detection unit 220 determines the detection result of each operand in each operation array according to the input data format and the exponent and mantissa of each operand in each operation array. For the E4M3 format data, when the exponent and mantissa of the floating point number are both 0, it is detected as zero; when the exponent and mantissa of the floating point number are both the maximum value, it is detected as non-number, and there is no infinity in the E4M3 format, so the detected infinity is always 0. For the data of the remaining floating point formats, when the exponent and mantissa of the floating point number are both 0, it is detected as zero, when the exponent of the floating point number is the maximum value and the mantissa is 0, it is detected as infinity, and when the exponent of the floating point number is the maximum value and the mantissa is not 0, it is detected as non-number. Then the detection result is filled into the matched pattern template to obtain the zero value distribution pattern, the infinity value distribution pattern and the non-number distribution pattern corresponding to each operation array respectively.

[0101] In the embodiment of the application, the zero value distribution pattern refers to the distribution of data with a value of zero in a certain number of floating point numbers. The infinity value distribution pattern refers to the distribution of data with a value of positive or negative infinity in a certain number of floating point numbers. The non-number distribution pattern refers to the distribution of data with a value of non-number in a certain number of floating point numbers.

[0102] In a specific example, when the total bit width of the detected special value is 64 bits, the preferred arrangement scheme can be:

[0103] When the data format is FP64, 2 FP64 data need to be detected, and the special value result of each FP64 data is 1 bit, which is located in the lowest 1 bit of the high and low 32 bits, and the values of the remaining bit fields are 0.

[0104] When the data format is FP32, 8 FP32 data need to be detected, and the special value result of each FP32 data is 1 bit, which is located in the low 1 bit of the 8 8-bit parts evenly divided from 64 bits, and the values of the remaining bit fields are 0.

[0105] When the data format is TF32, 16 TF32 data need to be detected, and the special value result of each TF32 data is 1 bit, which is located in the low 1 bit of the 16 4-bit parts evenly divided from 64 bits, and the values of the remaining bit fields are 0.

[0106] When the data format is FP16, 32 FP16 data need to be detected, and the special value result of each FP16 data is 1 bit, which is located in the low 1 bit of the 32 2-bit parts evenly divided from 64 bits, and the values of the remaining bit fields are 0.

[0107] When the data format is BF16, 32 BF16 data need to be detected, and the special value result of each BF16 data detection is 1 bit, which is located in the low 1 bit of the 32 2-bit bits divided by 64 bits, and the values of the remaining bit fields are 0.

[0108] When the data format is E5M2, 64 E5M2 data need to be detected, and the special value result of each E5M2 data detection is 1 bit, which is located in the 64 1-bit bits divided by 64 bits.

[0109] When the data format is E4M3, 64 E4M3 data need to be detected, and the special value result of each E4M3 data detection is 1 bit, which is located in the 64 1-bit bits divided by 64 bits.

[0110] For data format INT8, the special value result is all 0.

[0111] The sign processing unit 230 is used for performing bitwise XOR operation on the input two groups of sign results to obtain the sign of the product, and the special result judgment unit 240 judges the zero value distribution pattern, the infinite value distribution pattern and the non-number distribution pattern of each operation array through the pre-attached logic gate array, judges whether the output appears special result, and the special result includes infinite result and non-number result. The sign of the product is sent to the dot product operator 250, and the special result and the infinite sign are sent to the output processing unit 260. Among them, the pre-attached logic gate array is a hardware implementation for logical judgment of zero value, infinite value and non-number value of each element in the operation array, which can specifically include a comparator to determine the state of the value, a logic gate (such as an AND gate, an OR gate and a NOT gate) to combine the comparison result, a flip-flop to store these results, a multiplexer to select the input signal, a counter to count the number of special values, a state machine to control the judgment process, and a priority encoder to determine the first special value. In addition, it can also contain a custom logic module to meet the specific application requirements.

[0112] The multi-precision dot product operation hardware of the embodiment of the application comprises an input unpacking subunit, a special value detection unit, a sign processing unit, a special result judgment unit, a dot product operation unit and an output processing unit. The input unpacking subunit operates the array unpacking according to the input data format to form corresponding sign distribution patterns, exponent distribution patterns and mantissa distribution patterns, the special value detection unit determines and generates corresponding zero value distribution patterns, infinite value distribution patterns and non-number distribution patterns, the sign processing unit obtains the sign of the product of each pair of operands in the two operation arrays, the special result judgment unit obtains zero value judgment results, infinite value judgment results and non-number judgment results corresponding to the dot product results respectively, and then the dot product operation unit and the output processing unit obtain the output results.

[0113] The input unpacking subunit and the special value detection unit supporting multiple data formats and performing corresponding dot product operations according to the data formats and detection results can improve the calculation efficiency of the hardware, optimize the utilization rate of hardware resources, reduce energy consumption, and provide appropriate precision according to the needs of different application scenarios, thereby enhancing the generalization ability and stability of the hardware and adapting to the diversified needs of deep learning and scientific computing application scenarios.

[0114] Optionally, on the basis of each of the above embodiments, the input unpacking subunit 210 is further used to:

[0115] After the current filled mantissa is filled into the matching mantissa bit field in the matching pattern template in the data arrangement mode matching the input data format, if the exponent corresponding to the current filled mantissa is not 0, the previous bit of the highest bit of the mantissa bit field is set to 1; if the exponent corresponding to the current filled mantissa is 0, the mantissa bit field is left shifted by 1 bit to obtain the mantissa distribution pattern.

[0116] Specifically, when the data format is a floating point format, the mantissa bit field is extracted from the floating point format according to the IEEE 754 standard, and the mantissa is processed according to the exponent value. When the exponent value is not 0, 1 bits of 1 are supplemented before the highest bit of the extracted mantissa bit field, and when the exponent value is 0, the extracted mantissa bit field is left shifted by 1 bit. Then, the split mantissa is arranged according to the corresponding format according to the data format. When the exponent is not zero, the previous bit of the highest bit of the mantissa bit field is set to 1 to restore the integer 1 expressed implicitly in the IEEE 754 standard format; when the exponent is zero, the mantissa bit field is left shifted by one bit to match the definition of non-normalized numbers in the IEEE 754 standard. This processing method provides a unified processing flow, simplifies hardware design and software implementation, and helps to reduce the area and power consumption of the design.

[0117] Embodiment three

[0118] Figure 3 This is a schematic diagram of the structure of another multi-precision dot product operation hardware provided in the third embodiment of the present invention. This embodiment is a refinement of the dot product operator in the above embodiment, specifically including: an exponent processing unit, a shift value calculation unit, at least two multi-precision dot product calculation units, and a multi-precision addition calculation unit. Figure 3 Take multi-precision dot product computing hardware including two multi-precision dot product computing units as an example.

[0119] Correspondingly, such as Figure 3 As shown, the multi-precision dot product operation hardware includes: an input processing unit 310, an exponent processing unit 320, a shift value calculation unit 330, a multi-precision dot product calculation unit 340, a multi-precision dot product calculation unit 350, a multi-precision addition calculation unit 360 and an output processing unit 370, wherein:

[0120] The input processing unit 310 is configured to unpack the two operand arrays according to a preconfigured input data format, and obtain a sign distribution pattern, an exponent distribution pattern, and a mantissa distribution pattern corresponding to each operand array based on the unpacking result and the input data format; obtain a special value determination result of the dot product result, and calculate the sign of the product of each pair of operands in the two operand arrays;

[0121] An exponential processing unit 320 is configured to calculate an exponent of a dot product result obtained by performing a dot product calculation on two operands based on the exponential distribution pattern and the zero value distribution pattern of each operand;

[0122] a shift value calculation unit 330 for calculating a shift value of the product of each pair of operands in the two operand arrays according to the exponent of the dot product result;

[0123] The multiple-precision dot product calculation unit 340 and the multiple-precision dot product calculation unit 350 are each configured to calculate the product or partial product of at least one set of paired operands based on the sign distribution pattern and the mantissa distribution pattern of each operand array and the sign of the product of each paired operand; and to perform at least one shift alignment and at least one summation process on each product or partial product based on the input data format and / or the shift value of the product of each paired operand, to obtain an intermediate result of the dot product operation, wherein the intermediate result of the dot product operation includes the product of a set of paired operands, the partial product of a set of paired operands, or the sum of the products of two sets of paired operands.

[0124] A multi-precision addition calculation unit 360 is used to perform a sum operation on the intermediate dot product calculation results output by multiple multi-precision dot product calculation units to obtain a dot product result;

[0125] The output processing unit 370 is configured to round and normalize the point product mantissa according to the pre-configured standardized output format, rounding type and point product exponent, to obtain the normalized point product mantissa, normalized point product sign and normalized point product exponent as the normalized processing result, and to pack and output the normalized processing result according to the special value judgment result of the point product.

[0126] Specifically, the exponent processing unit 320 collects and analyzes the range and distribution of the exponent values from the exponent distribution pattern of each operation array, and adds the exponent values of the two groups of non-zero elements according to the zero value distribution pattern to obtain the point product exponent. The shift value calculation unit 330 calculates the difference value of the exponents of the paired operation arrays, and determines the shift amount according to the difference value to ensure the alignment of the exponents so as to correctly perform the mantissa multiplication and subsequent normalization processing.

[0127] The multi-precision point product calculation units 340 and 350 determine the numerical value of each element and the sign of the product result according to the sign distribution pattern, mantissa distribution pattern and sign of the product of each operation array, perform multiplication operation to obtain the product or partial product of the paired operation arrays, perform shift alignment on the product or partial product according to the input data format and / or the shift value of the product of each paired operation array to ensure the consistency of the exponents, and perform summation processing to obtain the intermediate result of the point product operation. The point product operation intermediate result includes the product of a group of paired operation arrays, i.e. the single product obtained by multiplying the elements at corresponding positions in the two input arrays; the partial product of a group of paired operation arrays, i.e. the partial product result of each pair of data, which finally needs to be added together to obtain the complete product result; or the sum of the products of two groups of paired operation arrays, i.e. if the product calculation of a part of operation array pairs has been completed and they are added together, this sum is the current intermediate result, which will be added with the subsequent partial product until all operation array pairs are processed.

[0128] The multi-precision addition calculation unit 360 performs summation operation on the point product operation intermediate results output by the multi-precision point product calculation units to realize the aggregation of all intermediate results and obtain the final result of the point product, which is delivered to the output processing unit 370.

[0129] The multi-precision dot product operation hardware of the embodiment of the application comprises an input processing unit, an index processing unit, a shift value calculation unit, at least two multi-precision dot product calculation units, a multi-precision addition calculation unit and an output processing unit. The index processing unit calculates the index of the dot product result according to the index distribution pattern and the zero value distribution pattern, the shift value calculation unit calculates the shift value of the product, the multi-precision dot product calculation unit calculates the product or the partial product according to the sign distribution pattern, the mantissa distribution pattern and the sign of the product, performs at least one shift alignment and at least one summation processing to obtain the dot product operation intermediate result, the multi-precision addition calculation unit performs summation operation on the dot product operation intermediate results output by the plurality of multi-precision dot product calculation units to obtain the dot product result. The input data group is processed by the at least two multi-precision dot product calculators, and the correctness of the dot product result of different data formats is ensured through the shift alignment and the summation processing, the multi-precision dot product operation hardware calculation efficiency and precision are improved, the resource utilization is optimized, the energy consumption is reduced, the calculation stability is enhanced, and a plurality of data formats can be adapted, mixed precision calculation is supported, so that the overall calculation performance is improved while the result accuracy is maintained.

[0130] Optionally, on the basis of each of the above embodiments, the index processing unit 320 is specifically configured to:

[0131] vector addition is performed on the index distribution pattern of each operation array to obtain an initial product index pattern;

[0132] bitwise OR operation is performed on the zero value distribution pattern of each operation array to obtain the detection result of whether the product of each pair of operation numbers in the two operation arrays is 0;

[0133] index correction is performed on the initial product index pattern according to the input data format and the detection result of whether the product of each pair of operation numbers is 0 to obtain a corrected product index pattern;

[0134] the maximum value of the product index in the product index corresponding to the product of each pair of operation numbers in the two operation arrays is compared according to the input data format and the corrected product index pattern;

[0135] the maximum value of the product index compared is determined as the index of the dot product result.

[0136] In the embodiment of the present application, the initial product exponent pattern can be understood as a pattern reflecting the exponent distribution of all products. When two floating-point numbers are multiplied, the exponent of the product is the sum of the exponents of the two numbers. Therefore, vector addition of the exponent distribution patterns of the two operation arrays is actually calculating the theoretical exponent value of each pair of corresponding elements after multiplication, and the distribution of the result constitutes the initial product exponent pattern. In a specific example, the 384 bits of exponents obtained after unpacking each input operation array are first vector added, and during the addition, each exponent is evenly divided into 32 parts with 12 bits each, and 32 12-bit adders are used to complete the vector addition to obtain the initial product exponent. During the vector addition, the data format is not concerned because the carry space has been reserved for each individual exponent when the exponent is unpacked.

[0137] Each bit in the zero-value distribution pattern represents whether an operand is zero. By comparing each bit of the two bitmaps through bitwise OR operation, if any bit is 1, the corresponding bit of the result is also 1, indicating that at least one operand is zero, so their product will also be zero. This method allows the calculation unit to determine in advance which product results will be zero without actually performing multiplication, thereby saving calculation resources and time.

[0138] According to the format of the input data and the detection result of whether the product is 0 obtained by the bitwise OR operation, the initial product exponent pattern is corrected, for example, special value processing is performed for the case where the product is zero, to obtain the corrected product exponent pattern. In a specific example, the special value detection results zero A and zero B are bitwise ORed to obtain the detection result of the product being 0 when the data format is floating-point format, and the initial product exponent is corrected accordingly. After correction, the corrected product exponent is obtained. When multiple precision correction is performed, the initial product exponent is corrected to 0 only when it is detected that the product is 0, otherwise no correction is performed. And it is necessary to decide how many bits in the initial product exponent to set to 0 according to the data format, for example, when the data format is E4M3 or E5M2 format, it is decided according to each bit in the detection result of the product being 0 whether to set the corresponding 6 bits of the initial product exponent evenly divided into 64 parts with 6 bits each to 0, and when the data format is the rest of the floating-point format, it is decided according to each even bit in the detection result of the product being 0 whether to set the corresponding 12 bits of the initial product exponent evenly divided into 32 parts with 12 bits each to 0.

[0139] According to the input data format and the modified product exponent pattern, the maximum value of the product exponents corresponding to the products of each pair of operands in the two operation arrays is compared by using the binary tree structure, and the maximum value obtained by the comparison is determined as the exponent of the dot product result, so that the order of magnitude of the dot product result can accurately reflect the maximum value in all products while meeting the data format limitation.

[0140] In a specific example, when the data format is E4M3 or E5M2 format, the modified product exponents are divided into 64 parts by 6 bits each, the binary tree structure is applied to the 64 exponents to compare the maximum value, and 6 bits of 0 are extended to the high bit of the maximum value to form a 12-bit maximum value result. When the data format is the rest of the floating point format, the modified product exponents are divided into 32 parts by 12 bits each, the binary tree structure is applied to the 32 exponents to compare the maximum value, and a 12-bit maximum value result is obtained. The maximum value is also used as the exponent of the dot product result.

[0141] Optionally, on the basis of the above embodiments, the shift value calculation unit 330 is specifically configured to:

[0142] According to the input data format, the exponent of the dot product result is expanded to obtain a vector format expansion exponent;

[0143] The vector format expansion exponent is subtracted from the modified product exponent pattern to obtain a product exponent difference pattern;

[0144] According to the input data format, the product exponent difference corresponding to the product of each pair of operands in the product exponent difference pattern is clamped to obtain a clamped product exponent difference pattern;

[0145] According to the input data format, a fixed shift offset is determined, and the fixed shift offset is added to the clamped product exponent difference pattern to obtain the shift value of the product of each pair of operands in the two operation arrays.

[0146] In the embodiment of the present application, the expansion processing can be understood as follows: the exponent of the dot product result is converted from its original format to a format suitable for use in a vector processing unit or a specific hardware architecture, for example, the exponent of the dot product result is copied multiple times to match the bit width of the product exponent pattern. The vector format expansion exponent can be understood as follows: in vector processing, it is required that the vector format of each vector operand is the same (i.e., the number of elements of each vector operand is the same, and the bit width of each element is the same), therefore, the exponent of the dot product result needs to be expanded to a format suitable for vector processing. For example, if the data format is E4M3 or E5M2, in the expansion processing, the low 6 bits of the exponent of the dot product result need to be copied 63 times to form a 384-bit vector format expansion exponent together with the low 6 bits of the dot product result; for other data formats, in the expansion processing, the exponent of the dot product result needs to be copied 31 times to form a 384-bit vector format expansion exponent together with the dot product result. The vector format expansion exponent is subtracted from the modified product exponent pattern element by element to obtain a product exponent difference pattern. This difference pattern represents the difference between the exponent of each product and the vector format expansion exponent.

[0147] In a specific example, when the data format is E4M3 or E5M2, the low 6 bits of the maximum value can be copied 64 times, and when the data format is the other formats, the complete 12 bits of the maximum value can be copied 32 times to form a vector format maximum value, and a vector subtraction similar to the vector addition unit is used to subtract the modified product exponent from the vector format maximum value to obtain a product exponent difference.

[0148] Clamping is an operation that limits the value within a predefined range to prevent the value from exceeding the representable range. According to the data format of the product of the operands, the product exponent difference corresponding to the product of each pair of operands in the product exponent difference pattern is divided by the unit number of bits, and it is determined whether each part of the product exponent difference is greater than the maximum value of the clamping range. If yes, the product exponent difference is set to the maximum value, otherwise the value of the product exponent difference is not modified. The clamped product exponent difference pattern ensures that the alignment and summation operations of the mantissa in the dot product operation can maintain the accuracy and efficiency of the calculation even when the exponent difference between the products is large. According to the accuracy requirement of the dot product operation, the product exponent difference is clamped to the required range to obtain a clamped product exponent difference, thereby reducing the bit width when aligning the products by shifting in the subsequent operation, which helps to reduce the chip area.

[0149] In one specific example, when the data format is E4M3 or E5M2, according to the dot product operation precision requirement, the product exponent difference needs to be clamped to the range of [0, 16], then the 384-bit product exponent difference is evenly divided into 64 parts by 6 bits, it is judged whether each part of the product exponent difference is greater than 16, if greater than 16, the product exponent difference is set to 16, otherwise the value of the product exponent difference is not modified. For another example, when the data format is the rest of the floating point format, according to the dot product operation precision requirement, the product exponent difference needs to be clamped to the range of [0, 32], then the 384-bit product exponent difference is evenly divided into 32 parts by 12 bits, it is judged whether each part of the product exponent difference is greater than 32, if greater than 32, the product exponent difference is set to 32, otherwise the value of the product exponent difference is not modified.

[0150] Due to the subsequent shift alignment stage of the product, in order to reduce the chip area, the same logic resource needs to be reused, and the effective data bit width of the product is different for different data formats. In order to reuse the same shift alignment logic, the corresponding fixed shift offset for different data formats is not the same, therefore, a shift value calculation unit is also needed to determine and add a fixed shift offset to the clamped product exponent difference pattern according to the input data format, to form the shift value of the product.

[0151] Optionally, on the basis of each of the above embodiments, the multi-precision dot product calculation unit 340 and the multi-precision dot product calculation unit 350 specifically include a plurality of mantissa multiplication sub-units, a floating point alignment sub-unit and a first-level summation sub-unit corresponding to each mantissa multiplication sub-unit, a partial product alignment sub-unit, a second-level summation sub-unit and a normalization output alignment sub-unit, wherein:

[0152] The mantissa multiplication sub-unit is used to perform signed multiplication calculation according to the allocated two input data and the sign of the product of the paired operands matched with the input data, to obtain two products or two partial products;

[0153] According to the different input data formats, a low-precision combination strategy, a high-precision segmentation strategy or no strategy is selected, the input data allocated to each mantissa multiplication sub-unit is determined according to the sign distribution pattern and the mantissa distribution pattern of each operation number, so that a group of product results of paired operands are calculated by the plurality of mantissa multiplication sub-units, two product results of two groups of paired operands are calculated by one mantissa multiplication sub-unit, or one product result of one group of paired operands is calculated by one mantissa multiplication sub-unit.

[0154] The floating-point alignment subunit is configured to perform or not perform shift alignment processing on two products or two partial products output by the matched mantissa multiplication subunit according to an input data format and a shift value of the product of each pair of operands, to obtain two aligned products or two partial products.

[0155] The first summation subunit is configured to perform summation operation on the two aligned products or the two partial products output by the matched floating-point alignment subunit, to obtain a first summation result.

[0156] The partial product alignment subunit is configured to perform or not perform shift alignment processing on each first summation result according to an input data format, to obtain a plurality of aligned first summation results.

[0157] The second summation subunit is configured to perform summation operation again on the plurality of first summation results input by the partial product alignment subunit, to obtain a second summation result.

[0158] The normalized output alignment subunit is configured to perform or not perform shift alignment processing on the second summation result according to an input data format, to obtain an intermediate result of the dot product operation.

[0159] Specifically, the mantissa multiplication subunit performs signed multiplication calculation on the product of two input data and a pair of operands matched with the input data according to the sign of the product, for example, the input is the product sign, the product shift value, the mantissa A and the mantissa B unpacked from the input operand array A and the input operand array B, to obtain two products or two partial products.

[0160] Preferably, the dot product operation unit can include 8 multi-precision dot product calculation units, each of which includes 4 mantissa multiplication subunits, a total of 32 mantissa multiplication subunits, responsible for completing 2 FP64 multiplications, 8 FP32 multiplications, 16 TF32 multiplications, 32 FP16 / BF16 multiplications and 64 E4M3 / E5M2 / INT8 multiplications. When the data format is FP64 and FP32, a low-precision combination strategy is used, and a plurality of mantissa multiplication subunits are used to calculate a product result together. When the data format is E4M3, E5M2 and INT8, a high-precision division strategy is used, so that one mantissa multiplication subunit calculates two product results. When the data format is TF32, FP16 and BF16, one mantissa multiplication subunit calculates one product result. However, when the data format is TF32, only half of the mantissa multiplication subunits are used. By integrating the low-precision combination and high-precision division strategies, the utilization rate of the multiplier resources and the calculation efficiency are improved.

[0161] The bit width of two inputs of each mantissa multiplication sub-unit is 16 bits. For the sake of clear description, the two inputs are called OP A and OP B hereinafter. In order to support the high-precision division strategy, two products are outputted when the data format is E4M3, E5M2 and INT8, and two partial products are outputted when the data format is the rest. For the sake of clear description, the two products or partial products are called PRODA and PRODB hereinafter. A preferred mantissa multiplication sub-unit uses the structure of booth encoding plus tree adder to realize signed number multiplication, and the structure uses OP B for booth encoding. The mantissa multiplication sub-unit supports two modes of 8b and 16b. When working in the mode of 16b, the mantissa multiplication sub-unit realizes the function of PRODA+PRODB<<8=OPA*OPB, and PRODA and PRODB are partial products with the bit width of 24 bits. When working in the mode of 8b, the mantissa multiplication sub-unit realizes the function of PRODA=OPA[7:0]*OPB[7:0] and PRODB=OPA[15:8]*OPB[15:8], and PRODA and PRODB are products with the bit width of 24 bits, in which the effective bit width is 16 bits and the high 8 bits are sign bits.

[0162] In order to support the low-precision combination strategy, a plurality of mantissa multiplication sub-units are combined according to the requirement to calculate a product together when the data format is different. In this application scenario, in order to reduce the logic resource, the mantissa multiplication sub-unit changes the effective bit width of one input OP A according to the requirement. Preferably, three mantissa multiplication sub-units are included, and the effective bit widths of OP A of the mantissa multiplication sub-units are 12 bits, 14 bits and 16 bits respectively, and the effective bit width of OP B is 16 bits. It should be noted that the effective bit width here is different from the actual bit width of the input port of the mantissa multiplication sub-unit. The actual bit width of the OP A port of the above three mantissa multiplication sub-units is 16 bits, and the effective bit width is not greater than the actual bit width. In actual operation, only the effective bit width participates in the calculation.

[0163] When the data format is FP64 format, the bit width of the mantissa of a single data after sign extension is 54 bits, which is greater than 16 bits, and the data needs to be segmented. The basis for segmentation is to minimize the effective bit width of the mantissa multiplication subunit OPA. Preferably, Al in the input operation array A is segmented into A10, A11, A12 and A13 in the order of 13 bits, 13 bits, 13 bits and 15 bits from low to high, and Bl in the input operation array B is segmented into B10, B11, B12 and B13 in the order of 11 bits, 11 bits, 16 bits and 16 bits from low to high. For A5 and B5, the same segmentation method as Al and Bl is used. After segmentation, the products of each segment of A and B are allocated to different mantissa multiplication subunits, and a total of 4x4x2 = 32 mantissa multiplication subunits are needed to calculate 2 FP64 multiplications (A1xB1 and A5xB5, respectively).

[0164] When the segments of A or B are allocated to the mantissa multiplication subunits, if the bit width of the segment is lower than the required bit width, 0 needs to be used to expand the high bits to meet the bit width requirement. When the product sign is negative, the result of the booth-encoded OPB needs to be bit-wise inverted to obtain the correct signed product. After calculating the partial products of each segment, the products need to be integrated to obtain the correct result. Taking A1xB1 as an example, the integration method of the FP64 product is as follows:

[0165] A1xB1 = A10xB10 + A10xB11<<11 + A10xB12<<22 + A10xB13<<38 + A11xB10<<13 + A11xB11<<24 + A11xB12<<35 + A11xB13<<51 + A12xB10<<26 + A12xB11<<37 + A12xB12<<48 + A12xB13<<64 + A13xB10<<39 + A13xB11<<50 + A13xB12<<61 + A13xB13<<77

[0166] When the data format is FP32 format, the mantissa of a single data after sign extension has a bit width of 25 bits, which is greater than 16 bits, and the data needs to be segmented. The basis for segmentation is to minimize the effective bit width of the mantissa multiplication subunit OPA. Preferably, Al in the input operation array A is divided into two segments A10 and A11 in the order of 13 bits and 12 bits from low to high, and B1 and B5 in the input operation array B are divided into two segments B10 and B11 in the order of 11 bits and 14 bits from low to high. For the remaining valid data in the input operation arrays A and B, the same segmentation method as Al and B1 is used. After segmentation, the products of each segment of A and B are allocated to different mantissa multiplication subunits, and a total of 4x8 = 32 mantissa multiplication subunits are needed to calculate the multiplication of 8 FP32. Preferably, 4 mantissa multiplication subunits in a single multi-precision dot product calculation unit are used to complete the multiplication of 1 FP32. After calculating the partial products of each segment, the products need to be integrated to obtain the correct result. Taking Al x B1 as an example, the integration method of FP32 product is as follows:

[0167] A1 x B1 = A10 x B10 + A10 x B11 << 11 + A11 x B10 << 13 + A11 x B11 << 24

[0168] When the data format is TF32 format, the mantissa of a single data after sign extension has a bit width of 12 bits, which is less than 16 bits, and a single mantissa multiplication subunit can be directly used for calculation. Calculating the multiplication of 16 TF32 format products only needs 16 mantissa multiplication subunits, and the utilization rate is only half. Preferably, in order to reduce power consumption, only half of the mantissa multiplication subunits in each multi-precision dot product calculation unit are used.

[0169] When the data format is BF16 or FP16 format, the mantissa of a single data after sign extension has a bit width of no more than 12 bits, which is also less than 16 bits, and a single mantissa multiplication subunit can be directly used for calculation. Calculating the multiplication of 32 BF16 or FP16 format products needs 32 mantissa multiplication subunits, and at this time each mantissa multiplication subunit in all multi-precision dot product calculation units is responsible for calculating the multiplication of 1 data.

[0170] Each multi-precision dot product calculation unit can contain one or more floating-point alignment subunits, such as a floating-point 8 / 16b alignment subunit. When the data format is E4M3, E5M2, FP16, BF16, and TF32, the two products or partial products output by the mantissa multiplication subunit are shifted and aligned according to the product shift value. When the data format is INT8, the products are already aligned, and there is no need to shift the two products output by the mantissa multiplier. When the data format is FP32 and FP64, PRODB needs to be shifted left by a fixed 8 bits.

[0171] Each multi-precision dot product calculation unit may include one or more first-level summation subunits, which calculate the sum of two products or partial products after shift alignment to obtain a first-level summation result.

[0172] Each multi-precision dot product calculation unit contains a partial product alignment subunit, such as the FP32 / FP64 partial product alignment subunit. When the data format is FP32 or FP64, one or more first-level summation results are each fixed-shift aligned. The fixed shift value of each first-level summation result is given by the above two integration methods. When the data format is other formats, there is no need to shift-align the first-level summation result, and one or more aligned first-level summation results are output.

[0173] Each multi-precision dot product calculation unit contains a second-level summation subunit, which sums one or more aligned first-level summation results to obtain a second-level summation result. This result is interpreted as a single product when the data format is FP32, as a partial product when the data format is FP64, and as a partial dot product result when the data format is other formats.

[0174] Each multi-precision dot product calculation unit includes a standardized output alignment subunit, such as the FP32 alignment subunit. When the data format is FP32, the secondary summation result is shift-aligned according to the product shift value. When the data format is other formats, no shift alignment is required. The result is interpreted as a single aligned product when the data format is FP32, as a partial product when the data format is FP64, and as a partial dot product result when the data format is other formats.

[0175] The low-precision combination strategy and high-precision splitting strategy of the mantissa multiplication subunits in the multi-precision dot product calculation unit use multiple mantissa multiplication subunits to combine and calculate a high-precision result in FP64 and FP32 formats. In the E4M2, E5M3 and INT8 formats, one mantissa multiplication subunit can be split into two multiplication subunits, thereby improving the resource sharing rate.

[0176] Further, on the basis of the above-mentioned embodiments, the dot product operator specifically further comprises a product exchange summation unit, wherein:

[0177] Correspondingly, the multiple-precision addition calculation unit, on the basis of implementing summation operation on the dot product operation intermediate results output by the multiple multiple-precision dot product calculation units to obtain a dot product result, is specifically configured to: when it is determined that the input data format belongs to the first type of data format, perform summation processing on the output results of the multiple multiple-precision dot product calculation units to obtain two products, and transmit the two products to the product exchange summation unit; and when it is determined that the input data format belongs to the second type of data format, perform summation processing on the output results of the multiple multiple-precision dot product calculation units to obtain a dot product result, and transmit the dot product result to the product exchange summation unit.

[0178] The product exchange summation unit is specifically configured to: when it is determined that the input data format belongs to the first type of data format, perform selective exchange on the two products according to the received two products and the shift values of the products of each pair of operands in the two operation arrays, and after shift alignment of the latter product according to the matched shift value of the product, calculate the sum of the two products as a dot product result; and when it is determined that the input data format belongs to the second type of data format, without exchange, shift alignment and summation, directly output the received dot product result. This module ensures that the format of the output dot product result is uniform regardless of the format of the input data, and the output dot product result of this unit is transmitted to the output processing unit.

[0179] In the embodiment of the application, the first type of data format can be specifically understood as: the input data format is FP64. The second type of data format can be specifically understood as: the input data format is not FP64. The multiple-precision addition calculation unit is configured to perform summation on the results output by the dot product operator according to the data format, when the data format is FP64, the summation result is two products of FP64, and the result is sent to the product exchange summation unit, when the data format is the remaining data format, the summation result is a plurality of input dot product results, and the dot product result is sent to the product exchange summation unit.

[0180] In a specific example, the multiple-precision addition calculation unit is configured to, when the data format is FP64, perform summation on part of the products output by the multiple-precision dot product calculation units 1-4 and the multiple-precision dot product calculation units 5-8 to obtain two products, and transmit the products to the product exchange summation unit for further processing. When the data format is the remaining format, perform summation on the input eight products or part of the dot product results to obtain a dot product result, and transmit the dot product result directly to the output processing unit via the product exchange summation unit. One preferred implementation of the multiple-precision addition calculation unit is to use a multiplexer and a tree-shaped adder.

[0181] The product exchange and summation unit is used to exchange the two product and product shift values according to the product shift values output by the exponent processing and shift value calculation module when the data format is FP64, so that the two product and product shift values after exchange are arranged in descending order of the exponent, and when the exponent values are the same, the two product and product shift values can be in any order. Then, the product with a smaller exponent (i.e., the latter product) is aligned according to the shift value. Finally, the sum of the two products after alignment is calculated to obtain the dot product result, which is directly transmitted to the output processing unit. When the data format is other formats, these operations are not performed, and the input is directly transmitted to the output processing unit. The product exchange and summation unit ensures that the dot product result output is uniform regardless of the data format.

[0182] In a specific example, when the data format is FP64, the exponent processing and shift value calculation module outputs two FP64 product shift values, referred to as the first and second product shift values. When the first product shift value is 0, the two FP64 products and shift values do not need to be exchanged, otherwise the two products and shift values need to be exchanged. Preferably, the exchange logic of the product and shift value can be implemented using a multiplexer. After exchange, the first product and product shift value is always the product with a larger exponent and its corresponding shift value (fixed as 0), and the second product and product shift value is always the product with a smaller exponent and its corresponding shift value (variable value). When the data format is other formats, no product exchange is performed. When the data format is FP64, after the exchange of the product and product shift value is completed, the second product is aligned according to the second product shift value, and then the first product and the aligned second product are summed to obtain the FP64 dot product result, which is directly transmitted to the output processing unit. When the data format is other formats, no shift alignment and summation are performed, and the input is directly transmitted to the output processing unit.

[0183] Specific application scenarios

[0184] With the rapid development of artificial intelligence large models, the computing power demand for matrix operation is significantly improved, and matrix operation hardware is an effective way to provide matrix computing power. In order to support different business needs in artificial intelligence tasks, such as neural network training and reasoning, the matrix computing power needs to support multiple precision data formats, generally using higher precision data formats in training, and using lower precision data formats in reasoning, so the basic hardware implementation (multiply-add operation) of matrix operation is required to support multiple precision data formats. In the prior art, a floating point calculation unit supporting multiple precision generally uses a low precision combination strategy to achieve high utilization of multipliers. These technical solutions only support floating point data format operations, do not support integer data format, and do not support low precision floating point data formats such as FP8, resulting in limited application scenarios and limited computing power in scenarios where low precision data format operations can be used. To solve the above problems, an embodiment of the present application provides a reconfigurable multi-precision dot product operation unit, Figure 4 is a reconfigurable multi-precision dot product operation unit suitable for an embodiment of the present application, as shown in Figure 4 , the hardware includes:

[0185] 1. Input unpacking and special value detection module

[0186] The input unpacking and special value detection module (equivalent to the input processing unit in the foregoing description) is used to unpack a plurality of floating point numbers or integers in the input operation array A and the input operation array B according to the data format (hereinafter referred to as DTYPE) and the operation mode used in the current operation, and detect special values in the plurality of floating point numbers in the input operation array A and the input operation array B, the special values including zero, infinity and non-number. The obtained two groups of signs, exponents, mantissas and special value results are distributed to the corresponding subsequent modules.

[0187] The dot product operation unit (equivalent to the multi-precision dot product operation hardware in the foregoing description) supports DTYPE including but not limited to floating point FP64, FP32, TF32, FP16, BF16, FP8 (including two formats FP8_E4M3, hereinafter referred to as E4M3, and FP8_E5M2, hereinafter referred to as E5M2) and integer INT8 format.

[0188] The unit supports operation modes including but not limited to FP64·FP64, FP32·FP32, TF32·TF32, FP16·FP16, BF16·BF16, E4M3·E4M3, E5M2·E5M2, E4M3·E5M2, E5M2·E4M3 and INT8·INT8 (wherein·represents dot product operation, i.e. the data format in front of·represents the format of the input operation array A, and the data format behind·represents the format of the input operation array B).

[0189] (1) unpacking result

[0190] The input operation array A and the input operation array B have the same data bit width, and the preferred value is 512 bits. Figure 5 is a schematic diagram of the index and arrangement of data in an input operation array suitable for an embodiment of the present application. As shown in Figure 5 , the index and arrangement of data in an input operation array are different when DTYPE is different.

[0191] When DTYPE is FP64, FP32, TF32, FP16, BF16, FP8, and INT8 format respectively, the 2, 8, 16, 32, 32, 64, and 64 floating point numbers or integers in the input operation array A and the input operation array B need to be unpacked. When DTYPE is FP64 or FP32, the floating point numbers that need to be unpacked only account for part of the input operation array A and the input operation array B. When DTYPE is FP64, the input operation arrays A and B each have 8 floating point FP64 format numbers, but only the 2 floating point FP64 format numbers in A and B need to be unpacked. The indexes of the two floating point numbers can be any two different indexes from 1 to 8, but the indexes used for A and B are the same. A preferred way is to use the floating point numbers with indexes 1 and 5. When DTYPE is FP32, the input operation arrays A and B each have 16 floating point FP32 format numbers, but only the 8 floating point FP32 format numbers in A and B need to be unpacked. The indexes of the two floating point numbers can be any 8 different indexes from 1 to 16, but the indexes used for A and B are the same. A preferred way is to use the floating point numbers with indexes 1, 2, 5, 6, 9, 10, 13, and 14. When DTYPE is the rest of the formats, all floating point numbers in the input operation arrays A and B need to be unpacked.

[0192] Figure 6 is a schematic diagram of the data required for unpacking of different data formats suitable for an embodiment of the present application. As shown in Figure 6 , the data required for unpacking when DTYPE is different is shown in the figure. The data at positions 3, 4, 7, 8, 11, 12, 15, and 16 in FP32 and positions 2, 3, 4, 6, 7, and 8 in FP64 in the figure represent data not used in the preferred scheme.

[0193] (2) index arrangement

[0194] Figure 7 is a schematic diagram of the preferred arrangement of the indexes split out for different data formats suitable for an embodiment of the present application. As shown in Figure 7As shown in FIG. 6, the total bit width of the split exponent is 384 bits. When DTYPE is a floating-point format, the exponent bit field is extracted from the floating-point format according to the IEEE 754 standard. When DTYPE is FP64, 2 FP64 data need to be split, and the exponent of each FP64 data is 11 bits, which is located in the lowest 11 bits of the high and low 192 bits, and the values of the remaining bit fields are 0. When DTYPE is FP32, 8 FP32 data need to be split, and the exponent of each FP32 data is 8 bits, which is located in the low 8 bits of the 8 48-bit parts into which the 384 bits are divided, and the values of the remaining bit fields are 0. The arrangement principle for other floating-point formats is described in Figure 7 . When DTYPE is INT8, the exponent value is all 0.

[0195] (3) Sign arrangement

[0196] Figure 8 is a schematic diagram of a preferred arrangement of a floating-point format sign suitable for embodiments of the present application. The preferred arrangement scheme of the split sign when DTYPE is different is as shown in Figure 8 . The total bit width of the split sign is 64 bits. When DTYPE is a floating-point format, the sign bit field is extracted from the floating-point format according to the IEEE 754 standard. When DTYPE is FP64, 2 FP64 data need to be split, and the sign of each FP64 data is 1 bit, which is located in the lowest 1 bit of the high and low 32 bits, and the values of the remaining bit fields are 0. When DTYPE is FP32, 8 FP32 data need to be split, and the sign of each FP32 data is 1 bit, which is located in the low 1 bit of the 8 8-bit parts into which the 64 bits are divided, and the values of the remaining bit fields are 0. The arrangement principle for other floating-point formats is described in Figure 8 . When DTYPE is INT8, the sign value is all 0.

[0197] (4) Exponent arrangement

[0198] Figure 9 is a schematic diagram of a preferred arrangement of a floating-point format sign suitable for embodiments of the present application. The preferred arrangement scheme of the split sign when DTYPE is different is as shown in Figure 9As shown, the total bit width of the split mantissa is 512 bits. When DTYPE is a floating point format, the mantissa bit field is extracted from the floating point format according to the IEEE 754 standard, and the mantissa is processed according to the exponent value. The specific processing principle is as follows: when the exponent value is not 0, 1 bits of 1 are added before the highest bit of the extracted mantissa bit field; when the exponent value is 0, the extracted mantissa bit field is left shifted by 1 bit. When DTYPE is FP64, 2 FP64 data need to be split and processed. The mantissa obtained by processing each FP64 data is 53 bits, which is located in the lowest 53 bits of the high and low 256 bits, and the values of the remaining bit fields are 0. When DTYPE is FP32, 8 FP32 data need to be split and processed. The mantissa obtained by processing each FP32 data is 24 bits, which is located in the low 24 bits of the 8 64 bits evenly divided from the 512 bits, and the values of the remaining bit fields are 0. For other floating point formats, the arrangement principle is explained in Figure 9 . For DTYPE as INT8, the mantissa value is a close arrangement of 64 INT8 data.

[0199] The unpacking result, exponent arrangement, sign arrangement and mantissa arrangement correspond to the functions of the input unpacking subunit described above.

[0200] (5) Special value detection and result arrangement (corresponding to the function of the special value detection unit described above)

[0201] When DTYPE is E4M3 format, the special value detection logic is as follows: when the exponent and mantissa values of the floating point number are both 0, it is detected as zero; when the exponent and mantissa values of the floating point number are both the maximum value, it is detected as non-number, and E4M3 format does not have infinity, so the detected infinity is always 0. When DTYPE is the remaining floating point format, the special value detection logic is as follows: when the exponent and mantissa values of the floating point number are both 0, it is detected as zero; when the exponent value of the floating point number is the maximum value and the mantissa value is 0, it is detected as infinity; when the exponent value of the floating point number is the maximum value and the mantissa value is not 0, it is detected as non-number.

[0202] Figure 10 is a schematic diagram of a floating point format special value detection result arrangement principle suitable for an embodiment of the present application. The preferred arrangement schemes of the three special value detection results when DTYPE is different are as follows: Figure 10As shown, the total bit width of the detected special value is 64 bits. When DTYPE is FP64, 2 FP64 data need to be detected, and the special value result of each FP64 data is 1 bit, which is located in the lowest 1 bit of the high and low 32 bits, and the values of the remaining bit fields are 0. When DTYPE is FP32, 8 FP32 data need to be detected, and the special value result of each FP32 data is 1 bit, which is located in the low 1 bit of the 8 bits divided in the 64 bits, and the values of the remaining bit fields are 0. For other floating point formats, the arrangement principle has been explained in Figure 10 . For DTYPE being INT8, the special value result is all 0.

[0203] 2. Symbol processing and special result judgment module

[0204] The symbol processing and special result judgment module (equivalent to the symbol processing unit and the special result judgment unit in the foregoing description) is used for performing bitwise XOR operation on the two groups of symbol results to obtain the symbol of the product, and judging whether a special result appears according to the two groups of special value results, the special result including an infinite result and a non-number result, and the corresponding symbol of the result when the result is infinite is also output. The symbol of the product is sent to the multi-precision dot product calculator (equivalent to the multi-precision dot product calculation unit in the foregoing description) and the multi-precision adder. The special result and the infinite symbol are sent to the result packaging module.

[0205] Figure 11 is a circuit schematic diagram of a preferred symbol processing and special result judgment module suitable for the embodiments of the present application, and a preferred circuit for calculating the symbol of the product and judging whether a special result appears is shown in Figure 11 , which can be realized by using some basic logic operations. In the figure, “zero A” and “zero B” respectively represent the zero result detected from the input operation array A and the input operation array B. Other methods such as “infinite A” and “non-number B” use the same method for analysis.

[0206] 3. Exponent processing and shift value calculation module

[0207] The exponent processing and shift value calculation module (equivalent to the exponent processing unit and shift value calculation unit mentioned above) is used to add the two sets of input exponential results element by element to obtain the exponent of each product, correct the exponent of the product based on whether the product is zero, and compare the exponents of these products to obtain the maximum exponent of the product. The exponent of each product is then subtracted from the maximum exponent of the product to obtain the exponent difference of each product. Finally, the shift value of each product is determined based on the exponent difference of these products and DTYPE. These product shift values ​​are sent to the multi-precision dot product unit and the FP64 product exchange and summation module. The maximum product exponent is sent as the exponent of the dot product result to the result packaging module.

[0208] Figure 12 FIG. 1 is a schematic diagram of an implementation method of a preferred exponential processing and shift value calculation module applicable to an embodiment of the present invention, such as Figure 12 As shown:

[0209] (1) A preferred implementation of the vector addition unit is as follows: 384-bit exponents A and B, obtained by unpacking the input operand arrays A and B, are first vector-added. During the addition, exponents A and B are each divided into 32 equal parts, each with 12 bits. Each part is summed using a 12-bit adder, resulting in a total of 32 12-bit adders for vector addition to obtain the initial product exponent. DTYPE is not a concern during vector addition because carry space is reserved for each individual exponent when the exponents are unpacked.

[0210] (2) The preferred implementation of the multi-precision correction unit is as follows: First, perform a bitwise OR operation on the special value detection results zero A and zero B to obtain a detection result of a product of 0 when DTYPE is a floating point format, and then correct the initial product index accordingly. After correction, a corrected product index is obtained. During multi-precision correction, the initial product index is corrected to 0 only when the product is detected to be 0, otherwise no correction is performed. It is also necessary to determine how many bits in the initial product index are set to 0 based on DTYPE. For example, when DTYPE is in E4M3 or E5M2 format, it is determined based on each bit in the detection result of the product being 0 whether to set the corresponding 6 bits after the initial product index is divided into 64 parts by 6 bits to 0. When DTYPE is in other floating point formats, it is determined based on each even bit (bits index starts from 0) in the detection result of the product being 0 whether to set the corresponding 12 bits after the initial product index is divided into 32 parts by 12 bits to 0.

[0211] (3) A preferred implementation of the multi-precision maximum value unit is as follows: When DTYPE is in E4M3 or E5M2 format, the modified product exponent is divided into 64 equal parts in units of 6 bits, and the 64 exponents are compared using a binary tree structure to obtain the maximum value. The high order bits of the maximum value are extended by 6 bits to form a 12-bit maximum value result. When DTYPE is in other floating-point formats, the modified product exponent is divided into 32 equal parts in units of 12 bits, and the 32 exponents are compared using a binary tree structure to obtain the maximum value, resulting in a 12-bit maximum value result. This maximum value is also used as the exponent of the dot product result.

[0212] (4) A preferred implementation of the expanded subtraction unit is: when DTYPE is E4M3 or E5M2, the lower 6 bits of the maximum value are copied 64 times, and when DTYPE is other formats, the full 12 bits of the maximum value are copied 32 times to form a vector format maximum value, and the corrected product exponent is subtracted from it using a vector subtraction similar to the vector addition unit to obtain the product exponent difference.

[0213] (5) A preferred implementation of the multi-precision clamping unit is as follows: based on the accuracy requirements of the dot product operation, the product exponent difference is clamped to a required range, and the clamped product exponent difference is obtained, thereby reducing the bit width of the subsequent shift alignment of the product, which helps to reduce the chip area. For example, when DTYPE is E4M3 or E5M2, based on the accuracy requirements of the dot product operation, the product exponent difference needs to be clamped to the range of [0, 16]. The 384-bit product exponent difference is then divided into 64 equal parts of 6 bits, and the product exponent difference of each part is determined to be greater than 16. If it is greater than 16, the product exponent difference is set to 16. Otherwise, the value of the product exponent difference is not modified. For another example, when DTYPE is other floating-point formats, the product exponent difference needs to be clamped to the range of [0, 32] according to the accuracy requirements of the dot product operation. In this case, the 384-bit product exponent difference is divided into 32 equal parts of 12 bits, and a check is made to see whether the product exponent difference of each part is greater than 32. If so, the product exponent difference is set to 32; otherwise, the value of the product exponent difference is not modified.

[0214] (6) A preferred implementation of the multi-precision shift value calculation unit is as follows: in order to reduce the area, the same logic resources need to be reused in the subsequent shift alignment stage of the product, and the effective data bit width of the product is different for different DTYPEs. In order to reuse the same shift alignment logic, different DTYPEs have different corresponding fixed shift offsets. Therefore, a multi-precision shift value calculation unit is also needed to add a shift offset to the clamped exponent difference to form the shift value of the product.

[0215] 4. Multi-precision dot product module

[0216] The multi-precision point productor module (equivalent to the multi-precision point product calculation unit in the foregoing text) is used to calculate the product or partial product of each mantissa according to the input two sets of mantissa and sign results, if the DTYPE is floating-point E4M3, E5M2, FP16 or BF16 (these formats are collectively referred to as floating-point 8b / 16b) format, then the variable value shift alignment of the floating-point product or partial product is performed according to the shift value of the product output by the exponent processing and shift value calculation module, otherwise the fixed value shift alignment of the floating-point product or partial product is performed according to the DTYPE, then the first summation is performed on the aligned product or partial product, the number of products or partial products is reduced by half, and the fixed value shift alignment of the partial product after the first summation is performed according to the DTYPE, this step is mainly to align the partial product when the DTYPE is FP32 and FP64, then the summation is performed on the aligned result again, the number of products or partial products is reduced to 1 / 4, finally, if the DTYPE is FP32, then the variable value shift alignment of the floating-point product is performed according to the shift value of the product output by the exponent processing and shift value calculation module, otherwise no shift alignment is needed, and the obtained result is the partial product when the DTYPE is FP64, the aligned product when the DTYPE is FP32, and the partial point product result when the DTYPE is the remaining format. The result is sent to the multi-precision adder module.

[0217] (1) Figure 13 is a preferred implementation schematic diagram of a multi-precision point productor module suitable for the embodiments of the present application, and a preferred implementation of the multi-precision point productor module is shown in Figure 13 , which inputs the product sign, the product shift value, the unpacked mantissa A and mantissa B of the input operation array A and the input operation array B. The output has different meanings when the DTYPE is different, and when the DTYPE is FP64, it is a partial product, when the DTYPE is FP32, it is an aligned product, and when the DTYPE is the remaining format, it is a partial point product result. The result is sent to the multi-precision adder module.

[0218] In a preferred embodiment of the present invention, 8 multi-precision dot product modules are included, each of which contains 4 mantissa multipliers (equivalent to the mantissa multiplication subunits mentioned above), totaling 32 mantissa multipliers, which are responsible for completing 2 FP64 multiplications, 8 FP32 multiplications, 16 TF32 multiplications, 32 FP16 / BF16 multiplications, and 64 E4M3 / E5M2 / INT8 multiplications. When DTYPE is FP64 and FP32, a low-precision combination strategy is used, and multiple mantissa multipliers jointly calculate a product result. When DTYPE is E4M3, E5M2, and INT8, a high-precision splitting strategy is used, so that one mantissa multiplier calculates two product results. When DTYPE is TF32, FP16, and BF16, one mantissa multiplier calculates one product result, but when DTYPE is TF32, only half of the mantissa multipliers are used. The technical solution proposed in the embodiment of the present invention improves the resource utilization and computing efficiency of the multiplier by integrating low-precision combination and high-precision segmentation strategies.

[0219] (2) Input bit width

[0220] Each mantissa multiplier has two 16-bit inputs. For clarity, these two inputs are referred to as OPA and OPB. To support high-precision segmentation strategies, two products are output when DTYPE is E4M3, E5M2, and INT8. For other DTYPE formats, two partial products are output. For clarity, these two products or partial products are referred to as PRODA and PRODB. A preferred mantissa multiplier uses a booth encoding plus a tree adder structure to implement signed number multiplication. This structure always uses OPB for booth encoding. The multiplier supports 8b and 16b modes. When working in 16b mode, the function implemented by the mantissa multiplier is PRODA+PRODB<<8=OPA*OPB. At this time, PRODA and PRODB are partial products, and the bit width is 24 bits. When working in 8b mode, the function implemented by the mantissa multiplier is PRODA=OPA[7:0]*OPB[7:0] and PRODB=OPA[15:8]*OPB[15:8]. At this time, PRODA and PRODB are products, and the bit width is also 24 bits, of which the effective bit width is 16 bits, and the high 8 bits are the sign bit.

[0221] To support low-precision combination strategies, multiple multipliers are combined as needed to calculate a single product at different DTYPEs. In this application scenario, to reduce logic resources, the mantissa multiplier changes the effective bit width of one of its input OPAs as needed.

[0222] Figure 14is a preferred implementation of the present application, and the preferred implementation of the present application contains a type of mantissa multiplier diagram, and the preferred implementation of the present application contains a type of mantissa multiplier as shown in Figure 14 As shown in the figure, the effective bit width of OP A and OPB is represented by "12bx16b", and in this preferred implementation, three types of mantissa multipliers are contained, and the effective bit width of OP A is 12 bits, 14 bits and 16 bits respectively, and the effective bit width of OPB is 16 bits. It should be noted that the effective bit width and the actual bit width of the input port of the mantissa multiplier are different, and the actual bit width of the above-mentioned three types of mantissa multiplier OP A port is 16 bits, and the effective bit width is not greater than the actual bit width, and in actual operation, only the effective bit width participates in the calculation.

[0223] When DTYPE is FP64 format, the bit width of the mantissa of a single data after sign extension is 54 bits, which is greater than 16 bits, and the data needs to be segmented. The basis for segmentation is to reduce the effective bit width of the mantissa multiplier OP A as much as possible. Figure 15 is a preferred segmentation mode diagram for the data format of the present application embodiment as FP64 format, as shown in Figure 15 As shown in the figure, A1 in the input operation array A is divided into A10, A11, A12 and A13 for a total of 4 segments in the order of 13 bits, 13 bits, 13 bits and 15 bits from low to high, and B1 in the input operation array B is divided into B10, B11, B12 and B13 for a total of 4 segments in the order of 11 bits, 11 bits, 16 bits and 16 bits from low to high. For A5 and B5, the same segmentation mode as A1 and B1 is used.

[0224] After segmentation, the products of each segment of A and B are distributed to different mantissa multipliers, and a total of 4x4x2=32 mantissa multipliers are needed to calculate 2 FP64 multiplications (A1xB1 and A5xB5 respectively).

[0225] Figure 16 is a preferred mantissa multiplier distribution scheme diagram for the data format of the present application embodiment as FP64 format, as shown in Figure 16 As shown in the figure, 16 mantissa multipliers in 4 multi-precision dot products are used to complete 1 FP64 multiplication.

[0226] It should be noted that when the segments of A or B are allocated to the mantissa multiplier, if the bit width of the segment is lower than the requirement, 0 extension is needed in the high bit to meet the bit width requirement. When the product sign is negative, the result of the booth encoded OPB needs to be bit-wise negated to get the correct signed product. After the partial products of each segment are calculated, the products need to be integrated to get the correct result. Take A1xB1 as an example to illustrate the integration method of the FP64 product:

[0227] A1xB1=A10xB10+A10xB11<<11+A10xB12<<22+A10xB13<<38+A11xB10<<13+A11xB11<<24+A11xB12<<35+A11xB13<<51+A12xB10<<26+A12xB11<<37+A12xB12<<48+A12xB13<<64+A13xB10<<39+A13xB11<<50+A13xB12<<61+A13xB13<<77

[0228] When the DTYPE is FP32 format, the bit width of the mantissa of a single data after sign extension is 25 bits, which is greater than 16 bits, so the data needs to be segmented. The basis of segmentation is to reduce the effective bit width of the mantissa multiplier OPA as much as possible. Figure 17 is a preferred segmentation method diagram of a data format of FP32 format suitable for embodiments of the present application, as shown in Figure 17 A1 in the input operation array A is segmented into A10 and A11 in the order of 13 bits and 12 bits from low to high, and B1 and B5 in the input operation array B are segmented into B10 and B11 in the order of 11 bits and 14 bits from low to high. For the remaining valid data in the input operation arrays A and B, the same segmentation method as A1 and B1 is used.

[0229] After segmentation, the products of each segment of A and B are allocated to different mantissa multipliers, and a total of 4x8=32 mantissa multipliers are needed to calculate 8 FP32 multiplications. Figure 18 is a preferred allocation scheme diagram of the mantissa multiplier suitable for a data format of FP32 format of embodiments of the present application, as shown in Figure 18 1 FP32 multiplication is completed using 4 mantissa multipliers in 1 multi-precision dot product. After the partial products of each segment are calculated, the products need to be integrated to get the correct result. Take A1xB1 as an example to illustrate the integration method of the FP32 product:

[0230] A1x B1 = A10x B10 + A10x B11 << 11 + A11x B10 << 13 + A11x B11 << 24

[0231] When DTYPE is TF32 format, the bit width of the mantissa of a single data after sign extension is 12 bits, which is less than 16 bits, and a single mantissa multiplier can be directly used for calculation. Calculating 16 products of TF32 format only needs 16 mantissa multipliers, and the utilization rate is only half. In the preferred implementation, in order to reduce power consumption, only half of the mantissa multipliers in each multi-precision dot product generator are used. Figure 19 is a preferred mantissa multiplier allocation scheme diagram suitable for a data format TF32 format of an embodiment of the application, as shown in Figure 19 Each dot product generator in the figure has two mantissa multipliers on the left in an off state.

[0232] When DTYPE is BF16 or FP16 format, the bit width of the mantissa of a single data after sign extension is not more than 12 bits, which is also less than 16 bits, and a single mantissa multiplier can be directly used for calculation. Calculating 32 products of BF16 or FP16 format needs 32 mantissa multipliers, and at this time each mantissa multiplier in all multi-precision dot product generators is responsible for calculating the product of 1 data, Figure 20 is a preferred mantissa multiplier allocation scheme diagram suitable for a data format BF16 or FP16 format of an embodiment of the application, and the preferred mantissa multiplier allocation scheme for the data format BF16 or FP16 format is as shown in Figure 20 .

[0233] In the preferred scheme of the application, each multi-precision dot product generator contains 4 floating point 8 / 16b alignment units (equivalent to the floating point alignment subunit in the foregoing), when DTYPE is E4M3, E5M2, FP16, BF16 and TF32, the two products or partial products output by the mantissa multiplier are shifted and aligned according to the product shift value, when DTYPE is INT8, the products are already aligned, and the two products output by the mantissa multiplier no longer need to be shifted, when DTYPE is FP32 and FP64, PRODB needs to be left shifted by 8 bits.

[0234] In the preferred scheme of the application, each multi-precision dot product generator contains 4 first-level summation units, which calculate the sum of the two products or partial products after shift alignment to obtain the first-level summation result.

[0235] In the preferred embodiment of the present application, each multi-precision dot product calculator contains one FP32 / FP64 partial product alignment unit (equivalent to the partial product alignment subunit in the foregoing text), which aligns the four primary summation results by fixed shifting when DTYPE is in FP32 or FP64 format, and the fixed shifting value of each primary summation result is given by the two integration modes described above. When DTYPE is in other formats, the primary summation results do not need to be aligned by shifting, and the four aligned primary summation results are output.

[0236] In the preferred embodiment of the present application, each multi-precision dot product calculator contains one secondary summation unit, which sums the four aligned primary summation results again to obtain one secondary summation result. When DTYPE is in FP32 format, the result is interpreted as a single product; when DTYPE is in FP64 format, the result is interpreted as a partial product; and when DTYPE is in other formats, the result is interpreted as a partial dot product result.

[0237] In the preferred embodiment of the present application, each multi-precision dot product calculator contains one FP32 alignment unit, which aligns the secondary summation result by shifting according to the product shift value when DTYPE is in FP32 format, and does not need to align by shifting when DTYPE is in other formats. The obtained result is interpreted as an aligned single product when DTYPE is in FP32 format, as a partial product when DTYPE is in FP64 format, and as a partial dot product result when DTYPE is in other formats.

[0238] 5. Multi-precision adder module

[0239] The multi-precision adder module (equivalent to the multi-precision addition calculation unit in the foregoing text) is used to sum the results output by the multi-precision dot product calculator according to DTYPE. When DTYPE is in FP64 format, the summation result is two FP64 products; and when DTYPE is in other data formats, the summation result is several input dot product results. The result is sent to the FP64 product exchange and summation module.

[0240] When DTYPE is in FP64 format, the partial products output by the multi-precision dot product calculators 1-4 and the multi-precision dot product calculators 5-8 are summed to obtain two products. When DTYPE is in other formats, the eight input products or partial dot product results are summed to obtain a dot product result. One preferred implementation is to use a multiplexer and a tree adder.

[0241] 6. FP64 product exchange and summation module

[0242] FP64 product exchange and sum module (equivalent to the product exchange sum unit in the foregoing text) is configured to selectively exchange two FP64 products and product shift values input according to product shift values output by the exponent processing and shift value calculation module, so that the two exchanged products and product shift values are arranged in descending order of the exponents, and when the exponents are the same, the two products can be arranged in any order. Then, the product with a smaller exponent is variably shifted and aligned according to the shift value. Finally, the sum of the two aligned products is calculated to obtain the dot product result when DTYPE is FP64. When DTYPE is any other format, these operations are not performed, and the input is directly transmitted to the output. This module ensures that the format of the output dot product result is uniform regardless of the format of DTYPE. The result is sent to the normalization and rounding module.

[0243] When DTYPE is FP64, the exponent processing and shift value calculation module outputs two FP64 product shift values, referred to as first and second product shift values. When the first product shift value is 0, the two FP64 products and shift values do not need to be exchanged, otherwise the two products and shift values need to be exchanged. Preferably, the exchange logic of the product and shift value can be implemented using a multiplexer. After exchange, the first product and product shift value is always the product with a larger exponent and its corresponding shift value (fixed as 0), and the second product and product shift value is always the product with a smaller exponent and its corresponding shift value (variable value). When DTYPE is any other format, product exchange is not performed.

[0244] When DTYPE is FP64, after the exchange of the product and product shift value is completed, the second product is variably shifted and aligned according to the second product shift value, and then the first product and the shifted second product are summed to obtain the dot product result in FP64. When DTYPE is any other format, no shift alignment and summation are performed. This module ensures that the format of the output dot product result is uniform regardless of the format of DTYPE.

[0245] 7. Normalization and rounding module

[0246] The normalization and rounding module (equivalent to the normalization and rounding unit in the foregoing text) is configured to normalize and round the input dot product result to obtain a dot product mantissa result and a sign that meet the requirements of the FP64 format.

[0247] The module first detects the number of head symbols of the input dot product result, performs initial normalization on the dot product result in combination with the index value, and then rounds the normalized result. In the preferred implementation, rounding to the 0 direction, rounding to the positive infinity direction, rounding to the negative infinity direction, and rounding to the nearest neighbor even number are supported. After rounding, the absolute value of the rounded result is taken, and normalization is performed again to obtain the dot product mantissa result and the sign that meet the FP64 format requirement. In the normalization process, the index value also needs to be modified accordingly according to the normalization result.

[0248] 8. Result packaging module

[0249] The result packaging module (equivalent to the result packaging unit described above) packages the result according to the input special result, sign, index, and dot product mantissa to form a dot product result in FP64 format.

[0250] When the special result detection module detects a special result such as infinity or NaN, the packaging module directly outputs the corresponding special result. If no special result is detected, the result is packaged according to the sign, index, and dot product mantissa result. In the packaging process, it is determined whether the index exceeds the representation range of the FP64 format. If the representation range is exceeded, the number is rounded to infinity or the maximum value according to the rounding mode. If the representation range is not exceeded, the normal value is output.

[0251] The reconfigurable multi-precision dot product operation unit proposed in the embodiment of the application can be automatically reconfigured into a dot product operation unit suitable for the data format set, and this reconfiguration is dynamically configurable on-chip. Regardless of the DTYPE used, the output dot product result is in FP64 format, ensuring high precision of the dot product result and reducing subsequent cumulative errors. A variety of DTYPE are supported, including but not limited to FP64, FP32, TF32, FP16, BF16, E4M3, E5M2, and INT8 formats. The supported DTYPE include high-precision data formats such as FP64 and FP32 for supercomputing, and low-precision data formats such as BF16 and FP8 for artificial intelligence, so the reconfigurable multi-precision dot product operation unit is particularly suitable for supercomputing and artificial intelligence application scenarios. The index arrangement scheme output by the input unpacking module can use the same adder to implement index summation for any DTYPE without any additional processing. The low-precision combination strategy and high-precision splitting strategy of the mantissa multiplier in the multi-precision dot product unit combine multiple mantissa multipliers to calculate a high-precision result when the FP64 and FP32 formats are used. When the E4M2, E5M3, and INT8 formats are used, one mantissa multiplier can be split into two multipliers, improving the resource sharing rate.

[0252] In addition, the reconfigurable multi-precision point product operation unit provided by the embodiment of the present application can change the supported DTYPE according to needs to support more or less data formats, can support more or less rounding modes according to needs, and the output data format is not limited to the FP64 format, and can be changed to other data formats according to needs. When the low-precision combination strategy is used, the input bit width of each mantissa multiplier and the allocation mode of the input data can be changed according to needs. In the reconfigurable multi-precision point product operation unit provided by the embodiment of the present application, any number of pipeline registers can be inserted to optimize the operation delay and improve the system working frequency. In the reconfigurable multi-precision point product operation unit provided by the embodiment of the present application, the data bit width in the input operation array A and the input operation array B is not limited to 512 bits, and can be changed to other bit widths according to the supported DTYPE. Correspondingly, the number of multi-precision point products is not limited to 8, and can be changed according to needs.

[0253] The reconfigurable multi-precision point product operation unit provided by the embodiment of the present application can multiplex the exponent summation, comparison, mantissa multiplication, alignment shift and summation calculation units under various DTYPE, reduce the design area and power consumption, and has a high hardware resource utilization rate; supports a wide range of data formats, not only supports low-precision floating point data formats, but also supports integer data formats, and has a wider application scenario, and can meet the application needs of supercomputing and artificial intelligence application scenarios; a single operation unit can complete point product operation of a large amount of data at a time, and improves the computing power and computing efficiency; the calculation precision is adjustable, and the calculation precision of the point product operation can be conveniently changed by adjusting the upper limit of the shift value in the product shift alignment, so as to meet the needs of different applications.

[0254] It should be understood that various forms of flow shown above can be used to reorder, add or delete steps. For example, each step described in the present application can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions of the present application can be achieved, and the present application is not limited herein.

[0255] The above detailed description does not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent replacement and improvement within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A multiple precision dot product operation hardware, comprising: The application relates to a dot product operation method and device. The input processing unit is used for unpacking two input operation arrays according to a pre-configured input data format, obtaining a symbol distribution pattern, an index distribution pattern and a mantissa distribution pattern corresponding to each operation array according to the unpacking result and the input data format, obtaining a special value judgment result of a dot product result, and calculating the sign of the product of each pair of operation numbers in the two operation arrays; The dot product operation unit is used for calculating the index of the dot product result and the shift value of the product of each pair of operation numbers in the two operation arrays according to the index distribution pattern of each operation array, calculating each product or partial product between each pair of operation numbers in the two operation arrays according to the symbol distribution pattern, the mantissa distribution pattern of each operation array and the sign of the product of each pair of operation numbers, and obtaining a plurality of dot product operation intermediate results after at least one shift alignment and at least one summation processing of each product or partial product according to the input data format and / or the shift value of the product of each pair of operation numbers; The output processing unit is used for rounding and normalizing the dot product mantissa according to a pre-configured standardized output format, a rounding type and the index of the dot product result, obtaining a normalized dot product mantissa, a normalized dot product sign and a normalized dot product index satisfying the standardized output format as a normalization processing result, and packing and outputting the normalization processing result according to the special value judgment result of the dot product result. The input processing unit specifically comprises an input unpacking subunit, a special value detection unit, a sign processing unit and a special result judgment unit, wherein: The input unpacking subunit is used for determining the unpacking quantity and unpacking position corresponding to each operation array according to the input data format after obtaining a plurality of data packets corresponding to each operation array, splitting the sign, the index and the mantissa of each operation number in each operation array in the data packets according to the unpacking quantity and the unpacking position, and filling the split sign, index and mantissa into a matched pattern template according to a data arrangement mode matched with the input data format to obtain the symbol distribution pattern, the index distribution pattern and the mantissa distribution pattern corresponding to each operation array; 2. The multiple precision dot product operation hardware of claim 1, wherein, The special value detection unit is used for determining the zero value detection result, the infinite value detection result and the non-number detection result of each operation number in each operation array according to the input data format and the index and the mantissa of each operation number in each operation array, and filling the zero value detection result, the infinite value detection result and the non-number detection result into a matched pattern template according to a data arrangement mode matched with the input data format to obtain the zero value distribution pattern, the infinite value distribution pattern and the non-number distribution pattern corresponding to each operation array; The sign processing unit is used for performing bitwise XOR operation on the symbol distribution pattern corresponding to each operation array to obtain the sign of the product of each pair of operation numbers in the two operation arrays. ​ ​ The special result judging unit is configured to perform logical judgment on the zero value distribution pattern, the infinite value distribution pattern and the non-number distribution pattern of each operation array by using the pre-stacked logic gate array, and obtain a zero value judgment result, an infinite value judgment result and a non-number judgment result corresponding to the dot product result respectively.

3. The multiple precision dot product operation hardware of claim 2, wherein, The input unpacking subunit is further configured to: After filling the split current filling mantissa into the matched mantissa bit field in the matched style template in the data arrangement mode matching the input data format, if the exponent corresponding to the current filling mantissa is not 0, the previous bit of the highest bit of the mantissa bit field is set to 1; if the exponent corresponding to the current filling mantissa is 0, the mantissa bit field is left shifted by 1 bit to obtain a mantissa distribution pattern.

4. The multiple precision dot product operation hardware of claim 2, wherein, The dot product operator specifically comprises: an exponent processing unit, a shift value calculation unit, at least two multi-precision dot product calculation units and a multi-precision addition calculation unit, wherein: The exponent processing unit is configured to calculate the exponent of the dot product result obtained after the dot product calculation of the two operation arrays according to the exponent distribution pattern and the zero value distribution pattern of each operation array; The shift value calculation unit is configured to calculate the shift value of the product of each pair of operation numbers in the two operation arrays according to the exponent of the dot product result; Each multi-precision dot product calculation unit is configured to calculate the product or partial product of at least one pair of operation numbers according to the sign distribution pattern, the mantissa distribution pattern of each operation array and the sign of the product of each pair of operation numbers, and then perform at least one shift alignment and at least one summation processing on each product or partial product according to the input data format and / or the shift value of the product of each pair of operation numbers, to obtain a dot product operation intermediate result, wherein the dot product operation intermediate result comprises the product of a pair of operation numbers, the partial product of a pair of operation numbers, or the sum of the products of two pairs of operation numbers; The multi-precision addition calculation unit is configured to perform summation operation on the dot product operation intermediate results output by the multi-precision dot product calculation units to obtain the dot product result.

5. The multiple precision dot product operation hardware of claim 4, wherein, The exponent processing unit is specifically configured to: perform vector addition on the exponent distribution pattern of each operation array to obtain an initial product exponent pattern; perform bit-by-bit OR operation on the zero value distribution pattern of each operation array to obtain a detection result of whether the product of each pair of operation numbers in the two operation arrays is 0; perform exponent correction on the initial product exponent pattern according to the input data format and the detection result of whether the product of each pair of operation numbers is 0 to obtain a corrected product exponent pattern; compare the maximum product exponent in the product exponents corresponding to the product of each pair of operation numbers in the two operation arrays according to the input data format and the corrected product exponent pattern; and determine the maximum product exponent obtained by comparison as the exponent of the dot product result.

6. The dot product operation hardware of claim 5, wherein, The shift value calculation unit is specifically configured to: perform extension processing on the exponent of the dot product result according to the input data format to obtain a vector format extended exponent; subtract the corrected product exponent pattern from the vector format extended exponent to obtain a product exponent difference pattern; and According to the input data format, the product exponent difference corresponding to the product of each pair of operation numbers in the product exponent difference pattern is clamped to obtain a clamped product exponent difference pattern; According to the input data format, a fixed shift offset is determined, and the fixed shift offset is added to the clamped product exponent difference pattern to obtain the shift value of the product of each pair of operation numbers in the two operation arrays.

7. The dot product operation hardware of claim 4, wherein, The multiple-precision dot product calculation unit specifically comprises multiple mantissa multiplication sub-units, a floating-point alignment sub-unit corresponding to each mantissa multiplication sub-unit, a first-level summation sub-unit, a partial product alignment sub-unit, a second-level summation sub-unit, and a normalized output alignment sub-unit, wherein: The mantissa multiplication sub-unit is configured to perform signed multiplication calculation on the basis of the assigned two input data and the sign of the product of the pair of operation numbers matched with the input data, to obtain two products or two partial products. According to the sign distribution pattern and the mantissa distribution pattern of each operation array, the input data assigned to each mantissa multiplication sub-unit is determined, so that a product result of a group of pair operation numbers is calculated by multiple mantissa multiplication sub-units, two product results of two groups of pair operation numbers are calculated by one mantissa multiplication sub-unit, or one product result of a group of pair operation numbers is calculated by one mantissa multiplication sub-unit. The floating-point alignment sub-unit is configured to perform or not perform shift alignment processing on the two products or two partial products output by the matched mantissa multiplication sub-unit according to the input data format and the shift value of the product of each pair of operation numbers, to obtain the aligned two products or two partial products. The first-level summation sub-unit is configured to perform summation operation on the aligned two products or two partial products output by the matched floating-point alignment sub-unit, to obtain a first-level summation result. The partial product alignment sub-unit is configured to perform or not perform shift alignment processing on each first-level summation result according to the input data format, to obtain multiple aligned first-level summation results. The second-level summation sub-unit is configured to perform summation processing again on the multiple first-level summation results input by the partial product alignment sub-unit, to obtain a second-level summation result. The normalized output alignment sub-unit is configured to perform or not perform shift alignment processing on the second-level summation result according to the input data format, to obtain an intermediate result of the dot product operation.

8. The multiple precision dot product operation hardware of claim 4, wherein, The dot product operation unit further comprises a product exchange summation unit. The multiple-precision addition calculation unit is specifically configured to: when it is determined that the input data format belongs to the first type of data format, perform summation processing on the output results of the multiple-precision dot product calculation units to obtain two products, and transmit the two products to the product exchange summation unit; and when it is determined that the input data format belongs to the second type of data format, perform summation processing on the output results of the multiple-precision dot product calculation units to obtain a dot product result, and directly transmit the dot product result to the product exchange summation unit. The product exchange summation unit is configured to, when the input data format is determined to belong to the first type of data format, selectively exchange the two products according to the shift values of the products of each pair of operation numbers in the two operation arrays received, and calculate the sum of the two products as the dot product result after the latter product is shifted and aligned according to the shift value of the matched product, and directly transmit the dot product result to the output processing unit. When the input data format is determined to belong to the second type of data format, no exchange, shift alignment and summation are performed, and the received dot product result is directly transmitted to the output processing unit.

9. The dot product operation hardware of claim 1, wherein, The output processing unit specifically includes a normalization and rounding unit and a result packaging unit, wherein: The normalization and rounding unit is configured to detect the number of sign bits contained in the input dot product result, and perform primary normalization on the dot product result according to the number of sign bits and the exponent of the input dot product result; perform rounding processing on the primary normalized dot product result according to a rounding type to obtain a rounding result; take the absolute value of the rounding result and perform secondary normalization to obtain a normalized dot product mantissa, a normalized dot product sign and a normalized dot product exponent that satisfy the standardized output format as a normalization processing result; The result packaging unit is configured to, when the dot product result is determined to be a special value according to the special value judgment result of the dot product result, package and output the special value; and when the dot product result is determined not to be a special value according to the special value judgment result of the dot product result, package the normalized dot product mantissa, the normalized dot product sign and the normalized dot product exponent to obtain a packaging result that satisfies the data representation range of the standardized output format and output the packaging result.

10. The dot product operation hardware according to any one of claims 1-9, characterized in that: The dot product operation hardware is capable of performing on-chip reconstruction of the configuration information in real time according to the configuration information of the input data format, the standardized output format and the rounding type matched with the current computing scenario, so as to meet the computing requirements of the current computing scenario.

Images