Multi-precision dot product operation hardware

Abstract

The invention discloses multi-precision dot product operation hardware which is characterized in that an input processing unit carries out unpacking according to an input data format and obtains a symbol, an index and a mantissa distribution pattern; judging a special value of a dot product result, and calculating a product symbol; the dot product arithmetic unit is used for calculating dot product indexes and product shift values according to the index distribution patterns; calculating a product or a partial product according to the symbol, the mantissa distribution pattern and the product symbol, performing shift alignment and summation according to an input data format and / or a shift value, and summing a dot product operation intermediate result to obtain a dot product result; and the output processing unit rounds and normalizes the dot product mantissa according to the standardized output format, the rounding type and the index of the dot product result, and packages and outputs the normalized processing result according to the special value of the dot product result. The hardware supports input data formats with various precisions, the application scene of the hardware is expanded, the design area and power consumption are reduced by multiplexing the dot product arithmetic unit, and the hardware resource utilization rate and the calculation efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of integrated chips, and in particular to multi-precision dot product operation hardware. Background Art

[0002] With the rapid development of large AI models, the computing power required for matrix operations has increased dramatically, and the hardwareization of matrix operations has become a key way to provide the required computing power. In order to meet the diverse business needs of AI tasks, the basic hardware for matrix operations, especially multiplication and addition operations, needs to support data formats of different precisions to adapt to different application scenarios.

[0003] In the prior art, the multiplication and addition units supporting multiple precision data formats achieve efficient utilization of basic multipliers through low-precision combination strategies. However, these technical solutions mainly focus on the operation of floating-point data formats, and fail to cover integer data formats and low-precision floating-point data formats such as FP8 (float point 8, 8-bit floating-point data format), which limits their application scope and restricts the use of computing power in scenarios where low-precision data formats are required, increases the complexity of calculations, causes waste of computing power resources, and reduces resource utilization and computing efficiency. Summary of the invention

[0004] The present invention provides a multi-precision dot product operation hardware to solve the problems that the data format supported by the dot product operation hardware has low precision, few applicable application scenarios, and low hardware resource utilization and computing efficiency.

[0005] According to one aspect of an embodiment of the present invention, a multi-precision dot product operation hardware is provided, comprising: an input processing unit, a dot product operator, and an output processing unit, wherein:

[0006] An input processing unit is used to unpack the two input operation arrays according to the pre-configured input data format, and obtain the sign distribution pattern, exponent distribution pattern and mantissa distribution pattern corresponding to each operation array according to the unpacking result and the input data format; obtain the special value judgment result of the dot product result, and calculate the sign of the product of each paired operand in the two operation arrays;

[0007] A dot product operator is used to calculate the exponent of the dot product result and the shift value of the product of each pair of operands in the two operation arrays according to the exponential distribution pattern of each operation array; calculate each product or partial product between each pair of operands in the two operation arrays according to the sign distribution pattern, the mantissa distribution pattern of each operation array and the sign of the product of each pair of operands; perform at least one shift alignment and at least one summation process on each product or partial product according to the input data format and / or the shift value of the product of each pair of operands to obtain multiple dot product operation intermediate results; perform a summation operation on the multiple dot product operation intermediate results to obtain a dot product result; the dot product result includes a dot product mantissa and a dot product sign;

[0008] The output processing unit is used to round and normalize the dot product mantissa according to the pre-configured standardized output format, rounding type and exponent of the dot product result; obtain the normalized dot product mantissa, normalized dot product sign and normalized dot product exponent that meet the standardized output format as the normalized processing result; and package and output the normalized processing result according to the special value judgment result of the dot product result.

[0009] The multi-precision dot product operation hardware of the embodiment of the present invention includes: an input processing unit unpacks and obtains a sign, exponent and mantissa distribution pattern according to an input data format; determines a special value of a dot product result and calculates a product sign; a dot product operator calculates a dot product exponent and a product shift value according to each exponent distribution pattern; calculates a product or a partial product according to the sign, mantissa distribution pattern and product sign, performs shift alignment and summation according to the input data format and / or shift value, and sums the dot product operation intermediate results to obtain a dot product result; an output processing unit rounds and normalizes the dot product mantissa according to a standardized output format, rounding type and the exponent of the dot product result, and packs and outputs a normalized processing result according to the special value of the dot product result. The hardware supports input data formats of multiple precisions, expands the application scenarios of the hardware, and helps reduce the design area and power consumption by reusing computing units, thereby improving hardware resource utilization and computing efficiency.

[0010] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0012] Figure 1 is a schematic diagram of the structure of a multi-precision dot product operation hardware provided according to the first embodiment of the present invention;

[0013] Figure 2 is a schematic diagram of the structure of another multi-precision dot product operation hardware provided according to the second embodiment of the present invention;

[0014] Figure 3 is a schematic diagram of the structure of yet another multi-precision dot product operation hardware provided according to Embodiment 3 of the present invention;

[0015] Figure 4 Schematic diagram of a reconfigurable multi-precision dot product operation unit applicable to an embodiment of the present invention;

[0016] Figure 5 It is a schematic diagram of indexing and arrangement of data in an input operation array applicable to an embodiment of the present invention;

[0017] Figure 6 This is a schematic diagram of data required for unpacking different data formats applicable to an embodiment of the present invention;

[0018] Figure 7 It is a schematic diagram of a preferred arrangement of indexes split from different data formats applicable to an embodiment of the present invention;

[0019] Figure 8 Schematic diagram of a floating point format symbol arrangement principle applicable to an embodiment of the present invention;

[0020] Fig. 9 It is a schematic diagram of a preferred arrangement of mantissas split from different data formats applicable to an embodiment of the present invention;

[0021] Fig.10 It is a schematic diagram of an arrangement principle of a floating-point format special value detection result applicable to an embodiment of the present invention;

[0022] Fig.11 It is a circuit diagram of a preferred symbol processing and special result judgment module applicable to the embodiment of the present invention;

[0023] Fig.12 It is a schematic diagram of a preferred implementation method of an exponential processing and shift value calculation module applicable to an embodiment of the present invention;

[0024] Fig.13 It is a schematic diagram of a preferred implementation method of a multi-precision dot product module applicable to an embodiment of the present invention;

[0025] Fig.14 It is a schematic diagram of a mantissa multiplier type included in a preferred implementation method applicable to an embodiment of the present invention;

[0026] Fig.15 It is a schematic diagram of a preferred segmentation method in which the data format is FP64 format applicable to an embodiment of the present invention;

[0027] Fig.16 It is a schematic diagram of a preferred mantissa multiplier allocation scheme for a data format of FP64 format applicable to an embodiment of the present invention;

[0028] Fig.17 It is a schematic diagram of a preferred segmentation method in which the data format is FP32 format applicable to an embodiment of the present invention;

[0029] Fig.18 It is a schematic diagram of an allocation scheme of a preferred mantissa multiplier in a data format of FP32 format applicable to an embodiment of the present invention;

[0030] Fig.19 It is a schematic diagram of a preferred mantissa multiplier allocation scheme for a data format of TF32 format applicable to an embodiment of the present invention;

[0031] Fig. 20 It is a schematic diagram of an optimal mantissa multiplier allocation scheme applicable to an embodiment of the present invention in a data format of BF16 or FP16 format. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] Embodiment 1

[0035] Figure 1 This is a schematic diagram of the structure of a multi-precision dot product operation hardware provided by the first embodiment of the present invention. This embodiment is applicable to dot product operations on data in multi-precision data formats, especially multi-precision data dot product operations in high-performance computing and artificial intelligence application scenarios. Figure 1 As shown, the multi-precision dot product operation hardware includes: an input processing unit 110, a dot product operator 120 and an output processing unit 130, wherein:

[0036] The input processing unit 110 is used to unpack the two input operation arrays according to the pre-configured input data format, and obtain the sign distribution pattern, exponent distribution pattern and mantissa distribution pattern corresponding to each operation array according to the unpacking result and the input data format; obtain the special value judgment result of the dot product result, and calculate the sign of the product of each pair of operands in the two operation arrays;

[0037] The dot product operator 120 is used to calculate the exponent of the dot product result and the shift value of the product of each pair of operands in the two operation arrays according to the exponent distribution pattern of each operation array; calculate each product or partial product between each pair of operands in the two operation arrays according to the sign distribution pattern, the mantissa distribution pattern of each operation array and the sign of the product of each pair of operands; perform at least one shift alignment and at least one summation process on each product or partial product according to the input data format and / or the shift value of the product of each pair of operands to obtain multiple dot product operation intermediate results; perform a summation operation on the multiple dot product operation intermediate results to obtain a dot product result; the dot product result includes a dot product mantissa and a dot product sign;

[0038] The output processing unit 130 is used to round and normalize the dot product mantissa according to the pre-configured standardized output format, rounding type and exponent of the dot product result; obtain the normalized dot product mantissa, normalized dot product sign and normalized dot product exponent that meet the standardized output format as the normalized processing result; and package and output the normalized processing result according to the special value judgment result of the dot product result.

[0039] In the embodiment of the present invention, the pre-configured input data format can be specifically understood as: the input data format supported by the multi-precision dot product operation hardware provided by the embodiment of the present invention for operation, which can specifically include: FP64 (double-precision floating point, 64 bits), FP32 (single-precision floating point, 32 bits), TF32 (Tensor Float 32, a floating point format for deep learning, while maintaining the dynamic range of FP32, the precision is similar to FP16), FP16 (half-precision floating point, 16 bits), BF16 (Brain Floating Point 16, 16-bit floating point for deep learning), FP8 (8-bit floating point) and INT8 (8-bit integer) formats. Among them, FP8 includes two formats: FP8_E4M3 (with 4-bit exponent and 3-bit mantissa, providing a smaller numerical range and higher precision), hereinafter referred to as E4M3; and FP8_E5M2 (with 5-bit exponent and 2-bit mantissa, providing a larger numerical range and smaller precision), hereinafter referred to as E5M2.

[0040] The multi-precision dot product operation hardware provided by the embodiment of the present invention supports high-precision data formats including FP64 and FP32, which are suitable for supercomputing. It also supports low-precision data formats such as BF16 and FP8, which meet the needs of artificial intelligence and make it suitable for application scenarios where supercomputing and artificial intelligence are integrated. In addition, the hardware allows the range of supported data formats to be dynamically adjusted according to specific application requirements, such as expanding to support more data formats, or streamlining supported data formats to optimize the performance of specific workloads.

[0041] Correspondingly, the operation modes supported by the multi-precision dot product operation hardware provided by the embodiment of the present invention may specifically include: FP64·FP64, FP32·FP32, TF32·TF32, FP16·FP16, BF16·BF16, E4M3·E4M3, E5M2·E5M2, E4M3·E5M2, E5M2·E4M3 and INT8·INT8. Wherein, · represents the dot product operation, that is, ·The data format in front represents the format of the input operation array A, ·The data format in the back represents the format of another input operation array B. The dot product is the result of multiplying the corresponding elements of two vectors and then summing them. The sign of the product is determined by the signs of the two operands, that is, a positive number multiplied by a positive number is a positive number, a negative number multiplied by a negative number is also a positive number, and a positive number multiplied by a negative number or a negative number multiplied by a positive number is a negative number.

[0042] Floating-point numbers consist of a sign bit, an exponent bit, and a mantissa bit. The sign bit is usually the leftmost bit and is used to indicate whether the value is positive or negative. The sign distribution pattern can be understood as follows: the distribution of the sign bits of all floating-point numbers in the input array reflects the distribution of positive and negative numbers in the input array. The exponent bit is used to indicate the range of the value. In the representation of floating-point numbers, the exponent bit determines the magnitude of the value. The exponent distribution pattern can be understood as follows: the distribution of the exponent bits of all floating-point numbers in the input array reflects the range and sparsity of the data in the input array. The mantissa bit is used to indicate the precision of the value. In the representation of floating-point numbers, the mantissa bit determines the accuracy of the value. The mantissa distribution pattern can be understood as follows: the distribution of the mantissa bits of all floating-point numbers in the input array reflects the accuracy of the data in the input array.

[0043] The input processing unit 110 unpacks a number of floating-point numbers or integers in each of the two input operation arrays according to the pre-configured input data format, and obtains a sign distribution pattern, an exponential distribution pattern, and a mantissa distribution pattern of the corresponding data format. Among them, the two operation arrays have the same data bit width, and one preferred value can be 512 bits. For floating-point type, the sign distribution pattern shows the distribution of positive and negative values, the exponential distribution pattern shows the value range and sparsity, and the mantissa distribution pattern shows the precision of the value. For integer type, since there is no exponent in its format and the value itself is already signed, its sign distribution and exponential distribution are both set to 0, and the mantissa distribution pattern shows the distribution of the integer value itself.

[0044] The input processing unit 110 detects special values ​​(zero, infinity, and non-number NaN) in a number of floating-point numbers in the input operand array to specially process the corresponding floating-point operations. For example, the result of multiplying any number with zero is zero, the result of multiplying any number with infinity is infinity, and the result of operations involving NaN is usually NaN. The special value judgment result of the dot product result is obtained, and the sign of the product of each pair of operands in the two operand arrays is calculated and transmitted to the dot product operator 120.

[0045] In floating-point multiplication, the exponent of the product of two floating-point numbers is the sum of the exponents of the two numbers. In addition, in order to align the decimal points, the mantissa needs to be shifted according to the difference in the exponents. Specifically, the shift value of the mantissa can be determined by calculating the difference in the exponents of each pair of operands in the two operand arrays.

[0046] In the dot product operation, the result of multiplying each paired operand (i.e., the corresponding elements in the two vectors) is called the product. In some data format configurations (such as FP32 and FP64), for the purpose of hardware multiplexing of the multiplier, the multiplier cannot be used to obtain the full product. In this case, the result output by the multiplier is called a partial product, indicating that it is part of the product. After obtaining the partial product, it is necessary to apply shifts and additions to multiple partial products to calculate the product. The final dot product result is the sum of all partial products or products added together.

[0047] The dot product operator 120 calculates each product or partial product between each pair of operands in the two operand arrays. Each product or partial product needs to be aligned with other partial products to ensure that the products or partial products have the same exponent and then added. Therefore, when performing large-scale calculations or processing data of different precisions, one or more shift alignment and summation processes are required to ensure that all products or partial products are correctly accumulated to obtain multiple dot product operation intermediate results. Finally, the multiple dot product operation intermediate results are summed to obtain the dot product result.

[0048] In addition, the dot product operator 120 can be reused under different data formats, that is, the same dot product operator can process different types of data, which not only reduces the demand for dedicated hardware resources, thereby reducing the design complexity and occupied area of ​​the integrated circuit, but also reduces power consumption, improves energy efficiency, and improves hardware resource utilization.

[0049] The standardized output format can specifically be: a limitation on the output format of the calculation result, generally a limitation on the mantissa, and can also include limitations on the sign and exponent. Preferably, the FP64 format is selected to output the dot product result to ensure high precision and accuracy of the calculation, effectively reducing the cumulative error that may occur during data processing. In addition, according to specific application requirements, the standardized output format can be converted into a variety of data formats including FP64, which not only ensures the accuracy of the hardware output results, but also improves the data availability and system adaptability.

[0050] Rounding type refers to the method of handling values ​​that exceed the representation range in floating-point operations, which may specifically include: rounding to zero, rounding to positive infinity, and rounding to negative infinity. Through the support of diverse rounding modes, the multi-precision dot product operation hardware proposed in the embodiment of the present invention can flexibly select appropriate rounding strategies according to different application scenarios and precision requirements. For example, selecting a rounding mode can reduce cumulative errors while maintaining numerical balance, or selecting a rounding mode to avoid systematic deviations of numerical values. In addition, the rounding modes supported by the hardware can be expanded or reduced according to actual usage requirements, further improving the adaptability and flexibility of the system.

[0051] In a specific example, the FP64 format is selected to output the dot product result. If the output result is not a special value, the output processing unit 130 normalizes the mantissa of the dot product result according to the pre-configured standardized output format, rounding type, and the exponent of the dot product result and the mantissa of the dot product result to eliminate the redundant sign bits in the mantissa of the dot product result when converted to the standardized output format. Then, the mantissa of the floating point number is rounded according to the rounding type. In particular, a certain number of shifted bits need to be retained when the mantissa is right-shifted, which are used for rounding after normalization. Finally, according to the special value judgment result of the dot product result, if the dot product result is zero, a zero value is directly output; if the dot product result is infinity or NaN, a corresponding special value is output; if the output result is not a special value, the normalized dot product mantissa, normalized dot product sign, and normalized dot product exponent after normalization and rounding are combined to form a normalized processing result, and these parts are packaged into a binary string, or converted into hexadecimal or other formats as needed and output.

[0052] The multi-precision dot product operation hardware of the embodiment of the present invention includes: an input processing unit unpacks and obtains a sign, exponent and mantissa distribution pattern according to an input data format; determines a special value of a dot product result and calculates a product sign; a dot product operator calculates a dot product exponent and a product shift value according to each exponent distribution pattern; calculates a product or a partial product according to the sign, mantissa distribution pattern and product sign, performs shift alignment and summation according to the input data format and / or shift value, and sums the dot product operation intermediate results to obtain a dot product result; an output processing unit rounds and normalizes the dot product mantissa according to a standardized output format, rounding type and the exponent of the dot product result, and packs and outputs a normalized processing result according to the special value of the dot product result. The hardware supports input data formats of multiple precisions, expands the application scenarios of the hardware, and helps reduce the design area and power consumption by reusing computing units, thereby improving hardware resource utilization and computing efficiency.

[0053] Optionally, based on the above embodiments, the output processing unit 130 may specifically include: a normalization and rounding unit, and a result packaging unit, wherein:

[0054] A normalization and rounding unit, used for detecting the number of sign bits contained in the input dot product result, and performing initial normalization on the dot product result according to the number of sign bits and the exponent of the input dot product result; performing rounding processing on the dot product result after the initial normalization according to the rounding type to obtain a rounded result; taking an absolute value of the rounded result, and normalizing it again to obtain a normalized dot product mantissa, a normalized dot product sign, and a normalized dot product exponent that meet the standardized output format as a normalization processing result;

[0055] A result packing unit is used to pack and output the special value when it is determined that the dot product result is a special value according to the special value judgment result of the dot product result; when it is determined that the dot product result is not a special value according to the special value judgment result of the dot product result, pack the normalized dot product mantissa, the normalized dot product sign and the normalized dot product exponent to obtain and output the packing result that satisfies the data representation range of the standardized output format.

[0056] In a specific example, the FP64 standardized output format is selected to output the dot product result, and the normalization and rounding unit is used to normalize and round the input dot product result to obtain the dot product mantissa result and sign that meet the FP64 format requirements. The normalization and rounding unit first detects the number of sign bits of the input dot product result, and performs initial normalization on the dot product result in combination with the exponent value. During the normalization process, the sign bit remains unchanged because the sign bit only indicates the positive or negative value of the value and does not involve the adjustment of the size or precision. The exponent value indicates the magnitude of the dot product result. During the normalization process, it is necessary to ensure that the exponent value is within the valid range, that is, the exponent value cannot exceed the maximum or minimum exponent value represented by the floating point number. If the exponent value exceeds the range, it needs to be adjusted. For example, if the exponent value exceeds the maximum exponent value, it means that the data exceeds the maximum range that can be represented by the standardized output format. At this time, it is necessary to decide to set the output to infinity or the maximum representable value of the standardized output format according to the rounding type. The mantissa indicates the effective digit part of the dot product result. During the normalization process, when the exponent value is greater than 0, it is necessary to ensure that the highest bit of the mantissa is 1. When the highest bit of the mantissa is less than 1, the mantissa needs to be shifted left until the highest bit is 1. For each left shift, the exponent value is reduced by 1. This is because shifting the mantissa left is equivalent to multiplying the value by 2, and the exponent value reduced by 1 is equivalent to dividing the value by 2, thus maintaining the invariance of the value. When the highest bit of the mantissa is greater than 1, the mantissa needs to be shifted right until the highest bit is 1. For each right shift, the exponent value is increased by 1. This is because shifting the mantissa right is equivalent to dividing the value by 2, and the exponent value increased by 1 is equivalent to multiplying the value by 2, thus maintaining the invariance of the value. During the normalization process, when the exponent value is less than or equal to 0, the mantissa needs to be shifted right and the exponent value is increased accordingly until the exponent value becomes 1. In this way, a denormalized value is generated. The dot product result is converted to a standard form through normalization to ensure that it meets the format requirements of the IEEE 754 standard floating-point number.

[0057] After that, the normalized result is rounded. Preferably, the supported rounding modes include rounding to 0, rounding to positive infinity, rounding to negative infinity, and rounding to the nearest even number. After rounding, the rounded result is taken in absolute value and normalized again to obtain the normalized dot product mantissa, normalized dot product sign, and normalized dot product exponent that meet the requirements of the FP64 format, and the normalized processing result is obtained by combining them. The rounding mode is used for rounding processing, the floating point number is adjusted to the required precision, and then the mantissa is converted to the unsigned number required by the FP64 format by taking the absolute value. Finally, normalization is performed again to adjust the result after taking the absolute value back to the standard form, thereby obtaining the standard mantissa.

[0058] The result packing unit packs the result according to the input special result, sign, exponent and dot product mantissa to form the dot product result in FP64 format. When the sign processing and special result judgment module detects that the result is a special result of infinity or non-number, the corresponding special result is packed and output. For example, in floating point representation, infinity is usually represented by all 1 exponent bits and all 0 mantissa bits, while non-number (NaN) is usually represented by all 1 exponent bits and non-all 0 mantissa bits. The output of these special values ​​ensures the accuracy and consistency of the calculation results.

[0059] If no special result is detected, it will be packed according to the normalized dot product mantissa, normalized dot product sign, and normalized dot product exponent. During the packing process, it will be determined whether the exponent exceeds the representation range of the FP64 format. Numbers that exceed the representation range will be rounded to infinity or the maximum value according to the rounding mode and packed for output. If it does not exceed the representation range, the result that meets the data representation range of the standardized output format will be packed and output, ensuring fast and accurate output when the dot product result is a special value, avoiding erroneous data processing and propagation, and improving the reliability of hardware calculations. At the same time, for non-special values, the normalized dot product sign, normalized dot product exponent, and normalized dot product mantissa are directly spliced ​​into the output FP64 format data, realizing fast data packing operations.

[0060] Optionally, based on the above embodiments, the multi-precision dot product operation hardware can perform on-chip reconstruction of configuration information in real time according to the configuration information of the input data format, standardized output format and rounding type that matches the current computing scenario to meet the computing requirements of the current computing scenario.

[0061] The multi-precision dot product operation hardware proposed in the embodiment of the present invention can change the working mode of the hardware in real time according to the actual task requirements, without the need for corresponding static configuration for each possible combination of data format and computing requirements.

[0062] In a specific example, the multi-precision dot product hardware can automatically identify the format of the input data and automatically adjust the working mode according to these formats. For example, if the input data is in FP16 format, the hardware will dynamically reconfigure the configuration on-chip, such as changing the bit width, precision, data path or other related parameters of the operation unit, to operate in a way that is most suitable for processing FP16 data and generate calculation results. The multi-precision dot product hardware can be dynamically configured and reconstructed on-chip in real time according to the specific needs of the current computing scenario, such as input data format, output format and rounding type, to adapt to different computing tasks, thereby improving the flexibility and adaptability of the hardware, and meeting diverse computing needs while maintaining efficient performance. Whether it is high-precision scientific computing or efficiency-oriented deep learning applications, it improves hardware resource utilization and computing efficiency.

[0063] Optionally, based on the above embodiments, the multi-precision dot product operation hardware can be inserted into any number of pipeline registers.

[0064] A pipeline register is a component used to temporarily store intermediate results. To ensure data relevance, all data paths from the original input to the original output must pass through the same number of pipeline registers to ensure synchronization and correct data transmission between different stages. By properly inserting any number of pipeline registers, complex combinational logic can be divided into shorter paths, reducing the number of logic levels and shortening data paths, thereby improving hardware throughput, optimizing computational latency, increasing system operating frequency, and improving hardware resource utilization and computing efficiency.

[0065] Embodiment 2

[0066] Figure 2 A schematic diagram of the structure of another multi-precision dot product operation hardware provided in the second embodiment of the present invention. This embodiment is a refinement of the input processing unit in the above embodiment, and specifically includes: an input unpacking sub-unit, a special value detection unit, a symbol processing unit and a special result judgment unit.

[0067] Correspondingly, such as Figure 2 As shown, the multi-precision dot product operation hardware includes: an input unpacking subunit 210, a special value detection unit 220, a symbol processing unit 230, a special result judgment unit 240, a dot product operator 250 and an output processing unit 260, wherein:

[0068] The input unpacking subunit 210 is used to determine the number of unpacking and the unpacking position corresponding to each operation array according to the input data format after obtaining multiple data packets corresponding to each operation array; according to the number of unpacking and the unpacking position, respectively split the sign, exponent and mantissa of each operand in each operation array in each data packet; according to the data arrangement method matching the input data format, fill the split sign, exponent and mantissa into the matching pattern template respectively, to obtain the symbol distribution pattern, exponent distribution pattern and mantissa distribution pattern corresponding to each operation array;

[0069] The special value detection unit 220 is used to determine the zero value detection result, infinite value detection result, and non-number detection result of each operand in each operand array according to the input data format and the exponent and mantissa of each operand in each operand array; fill each zero value detection result, infinite value detection result, and non-number detection result into the matching pattern template according to the data arrangement method matching the input data format, and obtain the zero value distribution pattern, infinite value distribution pattern, and non-number distribution pattern corresponding to each operand array respectively;

[0070] A symbol processing unit 230, configured to perform a bitwise XOR operation on the symbol distribution pattern corresponding to each operation array to obtain a sign of the product of each pair of operands in the two operation arrays;

[0071] A special result judgment unit 240 is used to perform logic judgment on the zero value distribution pattern, infinite value distribution pattern and non-number distribution pattern of each operation array through a pre-connected logic gate array to obtain a zero value judgment result, an infinite value judgment result and a non-number judgment result corresponding to the dot product result respectively;

[0072] The dot product operator 250 is used to calculate the exponent of the dot product result and the shift value of the product of each pair of operands in the two operation arrays according to the exponent distribution pattern of each operation array; calculate each product or partial product between each pair of operands in the two operation arrays according to the sign distribution pattern, the mantissa distribution pattern of each operation array and the sign of the product of each pair of operands; perform at least one shift alignment and at least one summation process on each product or partial product according to the input data format and / or the shift value of the product of each pair of operands to obtain multiple dot product operation intermediate results; perform a summation operation on the multiple dot product operation intermediate results to obtain a dot product result; the dot product result includes a dot product mantissa and a dot product sign;

[0073] The output processing unit 260 is used to round and normalize the dot product mantissa according to the pre-configured standardized output format, rounding type and exponent of the dot product result; obtain the normalized dot product mantissa, normalized dot product sign and normalized dot product exponent that meet the standardized output format as the normalized processing result; and package and output the normalized processing result according to the special value judgment result of the dot product result.

[0074] The input operation arrays have the same data bit width, and the preferred value is 512 bits. Specifically, after obtaining multiple data packets corresponding to each operation array, the input unpacking subunit 210 determines the number of unpacking and the unpacking position corresponding to each operation array according to the input data format. When the data formats are FP64, FP32, TF32, FP16, BF16, FP8 and INT8 formats, it is necessary to unpack the 2, 8, 16, 32, 32, 64 and 64 floating-point numbers or integers in each input operation array. When the data format is FP64 or FP32, the floating-point numbers that need to be unpacked only occupy a part of each input operation array. When the data format is FP64, each input operand has 8 floating-point FP64 format numbers, but only 2 floating-point FP64 format numbers in each input operand need to be unpacked. The indexes of these two floating-point numbers can be any 2 different indexes from 1 to 8, but the indexes used for each input operand are the same. A preferred method is to use floating-point numbers with indexes 1 and 5. When the data format is FP32, each input operand has 16 floating-point FP32 format numbers, but only 8 floating-point FP32 format numbers in each input operand need to be unpacked. The indexes of these two floating-point numbers can be any 8 different indexes from 1 to 16, but the indexes used for each input operand are the same. A preferred method is to use floating-point numbers with indexes 1, 2, 5, 6, 9, 10, 13 and 14. When the data format is other formats, all floating-point numbers in each input operand need to be unpacked.

[0075] In a specific example, when the total bit width of the split exponent is 384 bits, the total bit width of the split sign is 64 bits, and the total bit width of the split mantissa is 512 bits, after splitting the sign, exponent, and mantissa of each operand in each operation array, the corresponding sign distribution pattern, exponent distribution pattern, and mantissa distribution pattern are obtained according to the data arrangement method that matches the input data format.

[0076] The specific method of constructing the exponential distribution pattern is as follows: when the data format is a floating point format, the exponent bit field is extracted from the floating point format according to the IEEE 754 standard; when the data format is FP64, two FP64 data need to be split, and the exponent of each FP64 data split is 11 bits, which are located in the lowest 11 bits of the upper and lower 192 bits, and the values ​​of the remaining bit fields are 0.

[0077] When the data format is FP32, 8 FP32 data need to be split. The exponent of each FP32 data split is 8 bits, which are located in the lower 8 bits of the 8 48 bits divided equally from 384 bits, and the values ​​of the remaining bit fields are 0.

[0078] When the data format is TF32, 16 TF32 data need to be split. The exponent of each TF32 data split is 8 bits, which are located in the lower 8 bits of the 16 24 bits evenly divided from 384 bits, and the values ​​of the remaining bit fields are 0.

[0079] When the data format is FP16, 32 FP16 data need to be split. The exponent of each FP16 data split is 5 bits, which are located in the lower 5 bits of 32 12 bits evenly divided from 384 bits, and the values ​​of the remaining bit fields are 0.

[0080] When the data format is BF16, similar to FP16, 32 BF16 data need to be split. The exponent of each BF16 data split is 8 bits, which are located in the lower 8 bits of the 32 12 bits divided equally from 384 bits, and the values ​​of the remaining bit fields are 0.

[0081] When the data format is E5M2 (FP8_E5M2), 64 FP8 data need to be split. The exponent of each FP8 data split is 5 bits, which are located in the lower 5 bits of the 64 6 bits divided from 384 bits, and the values ​​of the remaining bit fields are 0.

[0082] When the data format is E4M3 (FP8_E4M3), the 64 FP8 data also need to be split. The exponent of each FP8 data split is 4 bits, which are located in the lower 4 bits of the 64 6 bits divided from the 384 bits, and the values ​​of the remaining bit fields are 0.

[0083] When the data format is INT8, the exponent value is all 0.

[0084] The symbol distribution pattern can be constructed specifically as follows: when the data format is a floating point format, the sign bit field is extracted from the floating point format according to the IEEE 754 standard; when the data format is FP64, two FP64 data need to be split, and the symbol split from each FP64 data is 1 bit, located at the lowest 1 bit of the upper and lower 2 32 bits, and the values ​​of the remaining bit fields are 0.

[0085] When the data format is FP32, 8 FP32 data need to be split. The symbol of each FP32 data split is 1 bit, which is located in the lower 1 bit of the 8 8 bits divided equally from 64 bits, and the values ​​of the remaining bit fields are 0.

[0086] When the data format is TF32, 16 TF32 data need to be split. The symbol of each TF32 data split is 1 bit, which is located in the lower 1 bit of the 16 4 bits evenly divided from 64 bits, and the values ​​of the remaining bit fields are 0.

[0087] When the data format is FP16, 32 FP16 data need to be split. The symbol of each FP16 data split is 1 bit, which is located in the lower 1 bit of the 32 2 bits evenly divided from 64 bits, and the values ​​of the remaining bit fields are 0.

[0088] When the data format is BF16, 32 BF16 data need to be split. The symbol split from each BF16 data is 1 bit, which is located in the lower 1 bit of the 32 2 bits evenly divided from 64 bits, and the values ​​of the remaining bit fields are 0.

[0089] When the data format is E5M2, 64 E5M2 data need to be split, and each symbol split from the E5M2 data is 1 bit, which is located in the 64 1 bits evenly divided from the 64 bits.

[0090] When the data format is E4M3, 64 E4M3 data need to be split, and the symbols split from each E4M3 data are 1 bits, which are respectively located in the 64 1 bits evenly divided from the 64 bits.

[0091] When the data format is INT8, the symbol value is all 0.

[0092] The specific method of constructing the mantissa distribution pattern is as follows: when the data format is a floating point format, the mantissa bit field is extracted from the floating point format according to the IEEE 754 standard; when the data format is FP64, two FP64 data need to be split and processed, and the mantissa obtained by processing each FP64 data is 53 bits, which are located in the lowest 53 bits of the upper and lower 256 bits, and the values ​​of the remaining bit fields are 0.

[0093] When the data format is FP32, 8 FP32 data need to be split and processed. The mantissa of each FP32 data is 24 bits, which are located in the lower 24 bits of the 8 64 bits divided equally from 512 bits, and the values ​​of the remaining bit fields are 0.

[0094] When the data format is TF32, 16 TF32 data need to be split and processed. The mantissa of each TF32 data is 11 bits, which are located in the lower 11 bits of the 16 32 bits divided equally from 512 bits, and the values ​​of the remaining bit fields are 0.

[0095] When the data format is FP16, 32 FP16 data need to be split and processed. The mantissa of each FP16 data is 11 bits, which are located in the lower 11 bits of the 32 16 bits divided equally from 512 bits, and the values ​​of the remaining bit fields are 0.

[0096] When the data format is BF16, 32 BF16 data need to be split and processed. The mantissa of each BF16 data is 8 bits, which are located in the lower 8 bits of the 32 16 bits divided equally from 512 bits, and the values ​​of the remaining bit fields are 0.

[0097] When the data format is E5M2, 64 E5M2 data need to be split and processed. The mantissa of each E5M2 data is 3 bits, which are located in the lower 3 bits of the 64 8 bits evenly divided from 512 bits, and the values ​​of the remaining bit fields are 0.

[0098] When the data format is E4M3, 64 E4M3 data need to be split and processed. The mantissa of each E4M3 data is 4 bits, which are located in the lower 4 bits of the 64 8 bits divided equally from 512 bits, and the values ​​of the remaining bit fields are 0.

[0099] When the data format is INT8, the mantissa value is a close arrangement of 64 INT8 data.

[0100] The special value detection unit 220 determines the detection result of each operand in each operand array according to the input data format and the exponent and mantissa of each operand in each operand array. For data in E4M3 format, when the exponent and mantissa values ​​of the floating point number are both 0, it is detected as zero; when the exponent and mantissa values ​​of the floating point number are both maximum values, it is detected as a non-number. There is no infinity in the E4M3 format, so the detected infinity is always 0. For data in other floating point formats, when the exponent and mantissa values ​​of the floating point number are both 0, it is detected as zero; when the exponent value of the floating point number is the maximum value and the mantissa value is 0, it is detected as infinity; when the exponent value of the floating point number is the maximum value and the mantissa value is not 0, it is detected as a non-number. Then fill the detection result into the matching style template to obtain the zero value distribution pattern, infinite value distribution pattern and non-number distribution pattern corresponding to each operand array respectively.

[0101] In the embodiment of the present invention, the zero value distribution pattern refers to the distribution of data with a value of zero in a certain number of floating point numbers. The infinite value distribution pattern refers to the distribution of data with a value of positive infinity or negative infinity in a certain number of floating point numbers. The non-number distribution pattern refers to the distribution of data with a value of non-number in a certain number of floating point numbers.

[0102] In a specific example, when the total bit width of the detected special value is 64 bits, the preferred arrangement scheme may be:

[0103] When the data format is FP64, two FP64 data need to be detected. The special value result detected for each FP64 data is 1 bit, which is located in the lowest 1 bit of the upper and lower 32 bits respectively, and the values ​​of the remaining bit fields are 0.

[0104] When the data format is FP32, 8 FP32 data need to be detected. The special value result detected for each FP32 data is 1 bit, which is located in the lower 1 bit of the 8 8 bits divided equally from the 64 bits, and the values ​​of the remaining bit fields are 0.

[0105] When the data format is TF32, 16 TF32 data need to be detected. The special value result detected for each TF32 data is 1 bit, which is located in the lower 1 bit of the 16 4 bits evenly divided from the 64 bits, and the values ​​of the remaining bit fields are 0.

[0106] When the data format is FP16, 32 FP16 data need to be detected. The special value result detected for each FP16 data is 1 bit, which is located in the lower 1 bit of 32 2 bits evenly divided from 64 bits, and the values ​​of the remaining bit fields are 0.

[0107] When the data format is BF16, 32 BF16 data need to be detected. The special value result detected by each BF16 data is 1 bit, which is located in the lower 1 bit of 32 2 bits evenly divided from 64 bits, and the values ​​of the remaining bit fields are 0.

[0108] When the data format is E5M2, 64 E5M2 data need to be detected, and the special value result detected by each E5M2 data is 1 bit, which is located in 64 1 bits evenly divided from the 64 bits.

[0109] When the data format is E4M3, 64 E4M3 data need to be detected, and the special value result detected by each E4M3 data is 1 bit, which is located in 64 1 bits evenly divided from the 64 bits.

[0110] When the data format is INT8, the special value result is all 0.

[0111] The symbol processing unit 230 is used to perform a bitwise XOR operation on the two sets of input symbol results to obtain the sign of the product, and in combination with the special result judgment unit 240, the zero value distribution pattern, the infinite value distribution pattern and the non-number distribution pattern of each operation array are logically judged through the pre-stacked logic gate array to judge whether a special result appears in the output, and the special result includes an infinite result and a non-number result. The sign of the product will be sent to the dot product operator 250, and the special result and the infinite sign will be sent to the output processing unit 260. Among them, the pre-stacked logic gate array is a hardware implementation for performing a logical judgment of zero value, infinite value and non-number value on each element in the operation array, and can specifically include a comparator to determine the state of the value, a logic gate (such as an AND gate, an OR gate and a NOT gate) to combine the comparison results, a trigger to store these results, a multiplexer to select the input signal, a counter to count the number of occurrences of the special value, a state machine to control the judgment process, and a priority encoder to determine the special value that appears first. In addition, a custom logic module can also be included to meet specific application requirements.

[0112] The multi-precision dot product operation hardware of the embodiment of the present invention includes: an input unpacking subunit, a special value detection unit, a sign processing unit, a special result judgment unit, a dot product operator, and an output processing unit. The input unpacking subunit unpacks the operation array according to the input data format to form a corresponding sign distribution pattern, an exponential distribution pattern, and a mantissa distribution pattern, the special value detection unit determines and generates the corresponding zero value distribution pattern, infinite value distribution pattern, and non-number distribution pattern, the sign processing unit obtains the sign of the product of each paired operand in the two operation arrays, the special result judgment unit obtains the zero value judgment result, infinite value judgment result, and non-number judgment result corresponding to the dot product result, respectively, and then the dot product operator and the output processing unit obtain the output result.

[0113] It supports input unpacking subunits and special value detection units in multiple data formats and performs corresponding point multiplication operations according to the data formats and detection results. It can improve the computing efficiency of the hardware, optimize the utilization of hardware resources, reduce energy consumption, and provide appropriate accuracy according to the needs of different application scenarios, thereby enhancing the generalization ability and stability of the hardware and adapting to the diverse needs of application scenarios such as deep learning and scientific computing.

[0114] Optionally, based on the above embodiments, the input unpacking subunit 210 is further used for:

[0115] After filling the split current filling mantissa into the matching mantissa bit field in the matching style template according to the data arrangement method matching the input data format, if the exponent corresponding to the current filling mantissa is not 0, the previous bit of the highest bit of the mantissa bit field is set to 1; if the exponent corresponding to the current filling mantissa is 0, the mantissa bit field is shifted left by 1 bit to obtain a mantissa distribution pattern.

[0116] Specifically, when the data format is a floating-point format, the mantissa bit field is extracted from the floating-point format according to the IEEE 754 standard, and the mantissa is processed according to the exponent value. When the exponent value is not 0, 1 bit of 1 is added before the highest bit of the extracted mantissa bit field. When the exponent value is 0, the extracted mantissa bit field is shifted left by 1 bit. Then, the split mantissa is arranged in the corresponding format according to the data format. When the exponent is not zero, the integer 1 implicitly expressed in the IEEE 754 standard format is restored by setting the previous bit of the highest bit of the mantissa bit field to 1; when the exponent is zero, the mantissa bit field is shifted left by one bit to match the definition of denormalized numbers in the IEEE 754 standard. This processing method provides a unified processing flow, simplifies hardware design and software implementation, and helps reduce the design area and power consumption.

[0117] Embodiment 3

[0118] Figure 3 A schematic diagram of the structure of another multi-precision dot product operation hardware provided in Embodiment 3 of the present invention. This embodiment is a refinement of the dot product operator in the above embodiment, and specifically includes: an exponent processing unit, a shift value calculation unit, at least two multi-precision dot product calculation units and a multi-precision addition calculation unit. Figure 3 Take multi-precision dot product computing hardware including two multi-precision dot product computing units as an example.

[0119] Correspondingly, such as Figure 3 As shown, the multi-precision dot product operation hardware includes: an input processing unit 310, an exponent processing unit 320, a shift value calculation unit 330, a multi-precision dot product calculation unit 340, a multi-precision dot product calculation unit 350, a multi-precision addition calculation unit 360 and an output processing unit 370, wherein:

[0120] The input processing unit 310 is used to unpack the two input operation arrays according to the pre-configured input data format, and obtain the sign distribution pattern, exponent distribution pattern and mantissa distribution pattern corresponding to each operation array according to the unpacking result and the input data format; obtain the special value judgment result of the dot product result, and calculate the sign of the product of each pair of operands in the two operation arrays;

[0121] An exponential processing unit 320, for calculating the exponent of a dot product result obtained after performing a dot product calculation on two operation arrays according to the exponential distribution pattern and the zero value distribution pattern of each operation array;

[0122] A shift value calculation unit 330, for calculating a shift value of the product of each pair of operands in two operand arrays according to an exponent of a dot product result;

[0123] The multiple-precision dot product calculation unit 340 and the multiple-precision dot product calculation unit 350, each of which is used to calculate the product or partial product of at least one group of paired operands according to the sign distribution pattern, the mantissa distribution pattern of each operand array and the sign of the product of each paired operand; according to the input data format and / or the shift value of the product of each paired operand, perform at least one shift alignment and at least one summation process on each product or partial product to obtain a dot product operation intermediate result, wherein the dot product operation intermediate result includes the product of a group of paired operands, the partial product of a group of paired operands, or the sum of the products of two groups of paired operands;

[0124] A multi-precision addition calculation unit 360, used for performing a summation operation on the intermediate results of dot product operations output by a plurality of multi-precision dot product calculation units to obtain a dot product result;

[0125] The output processing unit 370 is used to round and normalize the dot product mantissa according to the pre-configured standardized output format, rounding type and exponent of the dot product result; obtain the normalized dot product mantissa, normalized dot product sign and normalized dot product exponent that meet the standardized output format as the normalized processing result; and package and output the normalized processing result according to the special value judgment result of the dot product result.

[0126] Specifically, the exponent processing unit 320 collects and analyzes the range and distribution of the exponent value from the exponent distribution pattern of each operand array, and adds the exponent values ​​of the two groups of non-zero elements input in combination with the zero value distribution pattern to obtain the exponent of the dot product result. The shift value calculation unit 330 calculates the difference between the paired operand exponents, and determines the shift amount according to the difference to ensure that the exponents are aligned so as to correctly perform the mantissa multiplication and subsequent normalization processing.

[0127] The multiple precision dot product calculation unit 340 and the multiple precision dot product calculation unit 350 determine the value of each element and the sign of the product result according to the sign distribution pattern, the mantissa distribution pattern and the sign of the product of each operand array, perform multiplication operation, obtain the product or partial product of the paired operands, shift and align each product or partial product according to the input data format and / or the shift value of the product of each paired operand to ensure the consistency of the exponent, and perform summation processing to obtain the intermediate result of the dot product operation. Among them, the intermediate result of the dot product operation includes the product of a group of paired operands, that is, the single product obtained by multiplying the elements at corresponding positions in the two input arrays; the partial product of a group of paired operands, that is, the partial product result of each pair of data, which ultimately needs to be added together to obtain the complete product result; or the sum of the products of two groups of paired operands, that is, if the product calculation of a part of the operand pairs has been completed and they are added, this sum is the current intermediate result, which will be added to the partial product calculated subsequently until all operand pairs are processed.

[0128] The multi-precision addition calculation unit 360 performs a summation operation on the intermediate results of the dot product operations output by the multiple multi-precision dot product calculation units, aggregates all the intermediate results and obtains the final result of the dot product, and transmits the result to the output processing unit 370 .

[0129] The multi-precision dot product operation hardware of the embodiment of the present invention includes: an input processing unit, an exponent processing unit, a shift value calculation unit, at least two multi-precision dot product calculation units, a multi-precision addition calculation unit, and an output processing unit. The exponent processing unit calculates the exponent of the dot product result according to the exponential distribution pattern and the zero value distribution pattern, the shift value of the product is calculated by the shift value calculation unit, the multi-precision dot product calculation unit calculates the product or partial product according to the sign distribution pattern, the mantissa distribution pattern, and the sign of the product, performs at least one shift alignment and at least one summation process to obtain the dot product operation intermediate result, and the multi-precision addition calculation unit performs a summation operation on the dot product operation intermediate results output by multiple multi-precision dot product calculation units to obtain the dot product result. At least two multi-precision dot product calculators are used to process the operation of the input data group, and the correctness of the dot product results in different data formats is ensured through shift alignment and summation processing, thereby improving the hardware computing efficiency and accuracy of multi-precision dot product operations, optimizing resource utilization, reducing energy consumption, enhancing computing stability, and being able to adapt to a variety of data formats and support mixed-precision calculations, thereby improving the overall computing performance while maintaining the accuracy of the results.

[0130] Optionally, based on the above embodiments, the index processing unit 320 is specifically configured to:

[0131] Perform vector addition of the exponential distribution patterns of each operation array to obtain an initial product exponential pattern;

[0132] Performing a bitwise OR operation on the zero value distribution pattern of each operation array, and obtaining a test result of whether the product of each pair of operands in the two operation arrays is 0;

[0133] According to the input data format and the detection result of whether the product of each pairing operand is 0, the initial product exponential pattern is exponentially corrected to obtain a corrected product exponential pattern;

[0134] According to the input data format and the modified product index pattern, the maximum value of the product index among the product indexes corresponding to the products of each pair of operands in the two operation arrays is compared;

[0135] The maximum value of the product exponents obtained by comparison is determined as the exponent of the dot product result.

[0136] In the embodiment of the present invention, the initial product exponent pattern can be specifically understood as reflecting the exponential distribution of all products. When two floating-point numbers are multiplied, the exponent of the product is the sum of the exponents of the two numbers. Therefore, the exponential distribution patterns of the two operation arrays are vector-added, which is actually calculating the theoretical exponential value after each pair of corresponding elements are multiplied, and the distribution of the results constitutes the initial product exponent pattern. In a specific example, the 384-bit exponents obtained after unpacking the input operation arrays are first vector-added, and each exponent is divided into 32 parts in units of 12 bits during the addition. Each part is summed using a 12-bit adder, and a total of 32 12-bit adders are used to complete the vector addition to obtain the initial product exponent. The data format is not concerned when adding vectors, because carry space has been reserved for each individual exponent when the exponent is unpacked.

[0137] Each bit in the zero value distribution pattern represents whether an operand is zero. By comparing each bit of the two bitmaps through a bitwise OR operation, if any bit is 1, the corresponding bit of the result is also 1, indicating that at least one operand is zero, so their product will also be zero. This method allows the computing unit to predetermine which product results will be zero without actually performing multiplication operations, thereby saving computing resources and time.

[0138] According to the format of the input data and the detection result of whether the product is 0 obtained by bitwise OR operation, the initial product exponent pattern is exponentially corrected, for example, special value processing is performed on the case where the product is zero, and a corrected product exponent pattern is obtained. In a specific example, the special value detection results zero A and zero B are bitwise ORed to obtain a detection result that the product is 0 when the data format is a floating point format, and the initial product exponent is corrected accordingly. After correction, a corrected product exponent is obtained. In the case of multi-precision correction, the initial product exponent is corrected to 0 only when the product is detected to be 0, otherwise no correction is performed. And it is necessary to decide how many bits in the initial product exponent to set to 0 according to the data format. For example, when the data format is E4M3 or E5M2 format, it is decided whether to set the corresponding 6 bits after the initial product exponent is divided into 64 parts by 6 bits to 0 according to each bit in the detection result where the product is 0. When the data format is other floating point formats, it is decided whether to set the corresponding 12 bits after the initial product exponent is divided into 32 parts by 12 bits to 0 according to each even-numbered bit (bits index starts from 0) in the detection result where the product is 0.

[0139] According to the input data format and the corrected product exponent pattern, the maximum value of the product exponents in the product indices corresponding to the products of each paired operand in the two operation arrays can be compared through a binary tree structure, and the maximum value of the product exponent obtained by comparison can be determined as the exponent of the dot product result, ensuring that the magnitude of the dot product result can accurately reflect the maximum value of all products while meeting the limitations of the data format.

[0140] In a specific example, when the data format is E4M3 or E5M2 format, the modified product exponent is divided into 64 parts in units of 6 bits, and the 64 parts of the exponent are compared to obtain the maximum value by using a binary tree structure, and 6 bits of 0 are extended in the high position of the maximum value to form a maximum value result of 12 bits. When the data format is other floating point formats, the modified product exponent is divided into 32 parts in units of 12 bits, and the 32 parts of the exponent are compared to obtain the maximum value by using a binary tree structure to obtain the maximum value result of 12 bits. The maximum value is also used as the exponent of the dot product result.

[0141] Optionally, based on the above embodiments, the shift value calculation unit 330 is specifically configured to:

[0142] According to the input data format, the exponent of the dot product result is expanded to obtain a vector format expanded exponent;

[0143] Subtracting the corrected product index pattern from the vector format expanded index to obtain a product index difference pattern;

[0144] According to the input data format, the product exponent differences corresponding to the products of each pairing operand in the product exponent difference pattern are clamped to obtain a clamped product exponent difference pattern;

[0145] According to the input data format, a fixed shift offset is determined, and the fixed shift offset is added to the clamped product exponent difference pattern to obtain a shift value of the product of each paired operand in two operation arrays.

[0146] In the embodiment of the present invention, the expansion processing can be specifically understood as: converting the exponent of the dot product result from its original format to a format suitable for use in a vector processing unit or a specific hardware architecture, for example, copying the exponent of the dot product result multiple times to match the bit width of the product exponent pattern. The vector format expansion exponent can be specifically understood as: in vector processing, the vector format of each vector operand is required to be the same (that is, the number of elements of each vector operand is the same, and the bit width of each element is the same), so it is necessary to expand the exponent of the dot sum result to a format suitable for vector processing. For example, if the data format is E4M3 or E5M2, during the expansion processing, the lower 6 bits of the exponent of the dot product result need to be copied 63 times, and together with the lower 6 bits of the dot product result, a vector format expansion exponent of 384 bits is formed; for the remaining data formats, during the expansion processing, the exponent of the dot product result needs to be copied 31 times, and together with the dot product result, a vector format expansion exponent of 384 bits is formed. The vector format expansion exponent is subtracted element by element from the modified product exponent pattern to obtain a product exponent difference pattern. This difference pattern shows the difference between the exponent of each product and the vector format expanded exponent.

[0147] In a specific example, when the data format is E4M3 or E5M2, the lower 6 bits of the maximum value can be copied 64 times, and when the data format is other formats, the complete 12 bits of the maximum value can be copied 32 times to form a vector format maximum value, and the corrected product exponent can be subtracted therefrom using vector subtraction similar to the vector addition unit to obtain the product exponent difference.

[0148] Clamping is an operation that limits the value within a predefined range to prevent the value from exceeding the representable range. According to the data format of the product of the operands, the product exponent difference corresponding to the product of each paired operand in the product exponent difference pattern is evenly divided according to the number of unit bits, and it is determined whether the product exponent difference of each portion is greater than the maximum value of the clamping range. If so, the product exponent difference is set to the maximum value, otherwise the value of the product exponent difference is not modified. The clamped product exponent difference pattern ensures that even when the exponent difference between the products is large, the mantissa alignment and summation operations in the dot product operation can maintain the accuracy and efficiency of the calculation. According to the requirements for the accuracy of the dot product operation, the product exponent difference is clamped to the required range to obtain the clamped product exponent difference, thereby reducing the bit width of the subsequent shift alignment of the product, which helps to reduce the chip area.

[0149] In a specific example, when the data format is E4M3 or E5M2, according to the precision requirement of the dot product operation, the product exponent difference needs to be clamped to the range of [0, 16], then the 384-bit product exponent difference is divided into 64 parts with 6 bits, and it is determined whether the product exponent difference of each part is greater than 16. If it is greater than 16, the product exponent difference is set to 16, otherwise the value of the product exponent difference is not modified. For another example, when the data format is other floating point formats, according to the precision requirement of the dot product operation, the product exponent difference needs to be clamped to the range of [0, 32], then the 384-bit product exponent difference is divided into 32 parts with 12 bits, and it is determined whether the product exponent difference of each part is greater than 32. If it is greater than 32, the product exponent difference is set to 32, otherwise the value of the product exponent difference is not modified.

[0150] In the subsequent shift alignment stage of the product, in order to reduce the chip area, the same logic resources need to be reused, and the effective data bit width of the product is different for different data formats. In order to reuse the same shift alignment logic, different data formats have corresponding fixed shift offsets that are not the same. Therefore, it is also necessary to use a shift value calculation unit to determine according to the input data format and add a fixed shift offset to the clamped product exponent difference pattern to form a shift value of the product.

[0151] Optionally, based on the above embodiments, the multiple-precision dot product calculation unit 340 and the multiple-precision dot product calculation unit 350 specifically include multiple mantissa multiplication subunits, floating-point alignment subunits and a first-level summation subunit corresponding to each mantissa multiplication subunit, a partial product alignment subunit, a second-level summation subunit, and a standardized output alignment subunit, wherein:

[0152] A mantissa multiplication subunit, configured to perform a signed multiplication calculation according to two allocated input data and a sign of a product of a paired operand matching the input data, to obtain two products or two partial products;

[0153] According to different input data formats, a low-precision combination strategy, a high-precision segmentation strategy, or no strategy is selected, and input data allocated to each mantissa multiplication subunit is determined according to a sign distribution pattern and a mantissa distribution pattern of each operand array, so that a plurality of mantissa multiplication subunits jointly calculate a product result of a group of paired operands, a mantissa multiplication subunit independently calculates two product results of two groups of paired operands, or a mantissa multiplication subunit independently calculates one product result of a group of paired operands;

[0154] A floating point alignment subunit, configured to perform or not perform shift alignment processing on two products or two partial products output by the matching mantissa multiplication subunit according to an input data format and a shift value of the product of each paired operand, so as to obtain two aligned products or two partial products;

[0155] A first-level summing subunit is used to sum the two aligned products or two partial products output by the matching floating-point alignment subunit to obtain a first-level summing result;

[0156] The partial product alignment subunit is used to select whether to perform shift alignment processing on each first-level summation result or not according to the input data format, so as to obtain a plurality of aligned first-level summation results;

[0157] The secondary summation subunit is used to perform a secondary summation process on the multiple primary summation results input by the partial product alignment subunit to obtain a secondary summation result;

[0158] The standardized output alignment subunit is used to select whether to perform shift alignment processing on the secondary summation result or not according to the input data format to obtain the intermediate result of the dot product operation.

[0159] Specifically, the mantissa multiplication subunit performs signed multiplication calculation according to the sign of the product of two input data and the paired operands matching the input data, such as the input being the product sign, the product shift value, the mantissa A and the mantissa B unpacked from the input operation array A and the input operation array B, to obtain two products or two partial products.

[0160] Preferably, the dot product operator may include 8 multi-precision dot product calculation units, each of which includes 4 mantissa multiplication subunits, and a total of 32 mantissa multiplication subunits are responsible for completing 2 FP64 multiplications, 8 FP32 multiplications, 16 TF32 multiplications, 32 FP16 / BF16 multiplications, and 64 E4M3 / E5M2 / INT8 multiplications. When the data format is FP64 and FP32, a low-precision combination strategy is used, and multiple mantissa multiplication subunits jointly calculate a product result. When the data format is E4M3, E5M2, and INT8, a high-precision split strategy is used, so that one mantissa multiplication subunit calculates two product results. When the data format is TF32, FP16, and BF16, one mantissa multiplication subunit calculates one product result, but when the data format is TF32, only half of the mantissa multiplication subunits are used. By integrating low-precision combination and high-precision segmentation strategies, the multiplier resource utilization and computational efficiency are improved.

[0161] The bit width of the two inputs of each mantissa multiplication subunit is 16 bits. For the sake of clarity, these two inputs are referred to as OPA and OPB in the following text. In order to support high-precision segmentation strategies, when the data formats are E4M3, E5M2 and INT8, two products are output. When the data formats are other formats, two partial products are output. For the sake of clarity, these two products or partial products are referred to as PRODA and PRODB in the following text. A preferred mantissa multiplication subunit uses a booth encoding plus a tree adder structure to implement signed number multiplication. This structure fixedly uses OPB for booth encoding. The mantissa multiplication subunit supports both 8b and 16b modes. When working in 16b mode, the function implemented by the mantissa multiplication subunit is PRODA+PRODB<<8=OPA*OPB. At this time, PRODA and PRODB are partial products, and the bit width is 24 bits. When working in 8b mode, the function implemented by the mantissa multiplication subunit is PRODA=OPA[7:0]*OPB[7:0] and PRODB=OPA[15:8]*OPB[15:8]. At this time, PRODA and PRODB are products, and the bit width is also 24 bits, of which the effective bit width is 16 bits, and the high 8 bits are the sign bit.

[0162] In order to support low-precision combination strategies, in different data formats, multiple mantissa multiplication subunits are combined according to demand to jointly calculate a product. In this application scenario, in order to reduce logic resources, the mantissa multiplication subunit changes the effective bit width of one of the input OPAs according to demand. Preferably, a total of 3 types of mantissa multiplication subunits are included, and the effective bit widths of each OPA are 12 bits, 14 bits and 16 bits, respectively, and the effective bit width of OPB is 16 bits. It should be noted that the effective bit width here is different from the actual bit width of the input port of the mantissa multiplication subunit. The actual bit widths of the OPA ports of the above three types of mantissa multiplication subunits are all 16 bits, and the effective bit width is not greater than the actual bit width. In actual operations, only the effective bit width is involved in the calculation.

[0163] When the data format is FP64 format, the bit width of the mantissa of a single data after sign extension is 54 bits, which is greater than 16 bits, and the data needs to be segmented. The basis for segmentation is to minimize the effective bit width of the mantissa multiplication subunit OPA. Preferably, for A1 in the input operation array A, it is divided into 4 segments from low to high in the order of 13 bits, 13 bits, 13 bits and 15 bits, namely A10, A11, A12 and A13, and for B1 in the input operation array B, it is divided into 4 segments from low to high in the order of 11 bits, 11 bits, 16 bits, 16 bits, namely B10, B11, B12 and B13. For A5 and B5, the same segmentation method as A1 and B1 is used respectively. After the segmentation is completed, the products of each segment of A and B are assigned to different mantissa multiplication subunits, and a total of 4x4x2=32 mantissa multiplication subunits are required to complete the multiplication of 2 FP64 (A1xB1 and A5xB5 respectively). Preferably, 16 mantissa multiplication subunits in 4 multi-precision dot product calculation units can be used to complete the multiplication of 1 FP64.

[0164] When assigning segments of A or B to the mantissa multiplication subunit, if the bit width of the segment is lower than the requirement, it is necessary to use 0 extension in the high bit to meet the bit width requirement. When the product sign is negative, the result of booth encoding of OPB needs to be bitwise inverted to obtain the correct signed product. After calculating the partial products of each segment, these products need to be integrated to obtain the correct result. Taking A1xB1 as an example, the integration method of FP64 products is as follows:

[0165] A1×B1=A10×B10+A10×B11<<11+A10×B12<<22+A10×B13<<38+A11×B10<<13+A11×B11<<24+A11×B12<<35+A11×B13<<51+A12×B10<<26+A12×B11<<37+A12×B12<<48+A12×B13<<64+A13×B10<<39+A13×B11<<50+A13×B12<<61+A13×B13<<77

[0166] When the data format is FP32 format, the bit width of the mantissa of a single data after sign extension is 25 bits, which is greater than 16 bits, and the data needs to be segmented. The basis for segmentation is to minimize the effective bit width of the mantissa multiplication subunit OPA. Preferably, for A1 in the input operation array A, it is divided into 2 segments, A10 and A11, in the order of 13 bits and 12 bits from low to high, and for B1 and B5 in the input operation array B, it is divided into 2 segments, B10 and B11, in the order of 11 bits and 14 bits from low to high. For the remaining valid data in the input operation arrays A and B, the same segmentation method as A1 and B1 is used respectively. After the segmentation is completed, the products of each segment of A and B are assigned to different mantissa multiplication subunits, and a total of 4x8=32 mantissa multiplication subunits are required to calculate 8 FP32 multiplications. Preferably, 4 mantissa multiplication subunits in 1 multi-precision dot product calculation unit are used to complete 1 FP32 multiplication. After calculating the partial products of each segment, these products need to be integrated to obtain the correct result. Taking A1xB1 as an example, the integration method of FP32 products is as follows:

[0167] A1×B1=A10×B10+A10×B11<<11+A11×B10<<13+A11×B11<<24

[0168] When the data format is TF32 format, the bit width of the mantissa of a single data after sign extension is 12 bits, which is less than 16 bits, and can be directly calculated using a single mantissa multiplication subunit. Only 16 mantissa multiplication subunits are needed to calculate 16 products in TF32 format, and the utilization rate is only half. Preferably, in order to reduce power consumption, only half of the mantissa multiplication subunits are used in each multi-precision dot product calculation unit.

[0169] When the data format is BF16 or FP16, the bit width of the mantissa of a single data after sign extension does not exceed 12 bits, which is also less than 16 bits, and can be directly calculated using a single mantissa multiplication subunit. Calculating 32 products in BF16 or FP16 format requires 32 mantissa multiplication subunits. At this time, each mantissa multiplication subunit in all multi-precision dot product calculation units is responsible for calculating the product of 1 data.

[0170] Each multi-precision dot product calculation unit can contain one or more floating-point alignment subunits, such as a floating-point 8 / 16b alignment subunit. When the data format is E4M3, E5M2, FP16, BF16, and TF32, the two products or partial products output by the mantissa multiplication subunit are shifted and aligned according to the product shift value. When the data format is INT8, the product is already aligned, and there is no need to shift the two products output by the mantissa multiplier. When the data format is FP32 and FP64, PRODB needs to be shifted left by a fixed 8 bits.

[0171] Each multi-precision dot product calculation unit may include one or more first-level summation subunits, which calculate the sum of two products or partial products after shift alignment to obtain a first-level summation result.

[0172] Each multi-precision dot product calculation unit contains a partial product alignment subunit, such as an FP32 / FP64 partial product alignment subunit. When the data format is FP32 or FP64 format, one or more first-level summation results are each subjected to fixed shift alignment, and the fixed shift value of each first-level summation result is given by the above two integration methods. When the data format is other formats, there is no need to perform shift alignment on the first-level summation result, and one or more aligned first-level summation results are output.

[0173] Each multi-precision dot product calculation unit contains a secondary summation subunit, which sums one or more aligned primary summation results again to obtain a secondary summation result. This result is interpreted as a single product when the data format is FP32, as a partial product when the data format is FP64, and as a partial dot product result when the data format is other formats.

[0174] Each multi-precision dot product calculation unit contains a standardized output alignment subunit, such as the FP32 alignment subunit. When the data format is FP32, the secondary summation result is shifted and aligned according to the product shift value. When the data format is other formats, no shift alignment is required. The result is interpreted as a single product after alignment when the data format is FP32, as a partial product when the data format is FP64, and as a partial dot product result when the data format is other formats.

[0175] The low-precision combination strategy and high-precision splitting strategy of the mantissa multiplication subunit in the multi-precision dot product calculation unit use multiple mantissa multiplication subunits to combine and calculate a high-precision result in FP64 and FP32 formats. In E4M2, E5M3 and INT8 formats, one mantissa multiplication subunit can be split into two multiplication subunits, which improves the resource sharing rate.

[0176] Further, based on the above embodiments, the dot product operator may further include a product exchange summation unit, wherein:

[0177] Correspondingly, the multiple-precision addition calculation unit, on the basis of implementing the summation operation on the intermediate results of the dot product operation output by the multiple multiple-precision dot product calculation units to obtain the dot product result, is specifically used to: when it is determined that the input data format belongs to the first type of data format, sum the output results of each multiple-precision dot product calculation unit to obtain two products, and transmit the two products to the product exchange summation unit; and when it is determined that the input data format belongs to the second type of data format, sum the output results of each multiple-precision dot product calculation unit to obtain the dot product result, and transmit the dot product result to the product exchange summation unit;

[0178] The product exchange and summation unit is specifically used to: when it is determined that the input data format belongs to the first type of data format, according to the shift values ​​of the two products and the products of each paired operand in the two operation arrays, selectively exchange the two products, and after shifting and aligning the latter product according to the shift value of the matching product, calculate the sum of the two products as the dot product result; and when it is determined that the input data format belongs to the second type of data format, no exchange, shift alignment and summation are performed, and the received dot product result is directly output. This module ensures that no matter what format the input data is in, the format of the output dot product result is unified, and the output dot product result of this unit is transmitted to the output processing unit.

[0179] In the embodiment of the present invention, the first type of data format can be specifically understood as: the input data format is FP64. The second type of data format can be specifically understood as: the input data format is not FP64. The multi-precision addition calculation unit is used to sum the results output by the dot product operator according to the data format. When the data format is FP64, the summation result is the product of two FP64s and the result is sent to the product exchange summation unit. When the data format is other data formats, the summation result is the dot product result of several inputs, and the dot product result is sent to the product exchange summation unit.

[0180] In a specific example, a multiple precision addition calculation unit is used to sum the partial products output by multiple precision dot product calculation units 1 to 4 and multiple precision dot product calculation units 5 to 8 when the data format is FP64, to obtain 2 products, and transmit the products to the product exchange summation unit for further processing. When the data format is other formats, the 8 input products or partial dot product results are summed to obtain the dot product result, and the dot product result is directly transmitted to the output processing unit via the product exchange summation unit. A preferred implementation of the multiple precision addition calculation unit is to use a multiplexer and a tree adder.

[0181] The product exchange summation unit is used to selectively exchange the two input products and product shift values ​​according to the product shift value output by the exponent processing and shift value calculation module when the data format is FP64, so that the two exchanged products and product shift values ​​are arranged in descending order according to the exponent. When the exponent values ​​are the same, the two can be in any order. After that, the product with a smaller exponent (i.e., the next product) is variably shifted and aligned according to the shift value. Finally, the sum of the two aligned products is calculated to obtain the dot product result, and the dot product result is directly transmitted to the output processing unit. When the data format is other formats, these operations are not performed and it is directly transmitted to the output processing unit. The product exchange summation unit ensures that the output dot product result format is unified regardless of the data format.

[0182] In a specific example, when the data format is FP64 format, the exponent processing and shift value calculation module will output the shift values ​​of two FP64 products, respectively referred to as the first and second product shift values. When the first product shift value is 0, there is no need to exchange the two FP64 products and shift values, otherwise the two products and shift values ​​need to be exchanged. Preferably, the exchange logic of the product and shift value can be implemented using a multiplexer. After the exchange, the first product and the product shift value are always the product with a larger exponent value and its corresponding shift value (fixed to 0), and the second product and the product shift value are always the product with a smaller exponent value and its corresponding shift value (variable value). When the data format is the other formats, the product exchange will not be performed. When the data format is FP64 format, after completing the exchange of the product and the product shift value, the second product is variable shifted and aligned according to the second product shift value, and then the first product and the shifted second product are summed to obtain the dot product result of FP64, and the dot product result is directly transmitted to the output processing unit. When the data format is the rest format, shift alignment and summing are not performed, and the input is directly transmitted to the output processing unit.

[0183] Specific application scenarios

[0184] With the rapid development of large artificial intelligence models, the demand for matrix computing power has increased significantly, and the hardwareization of matrix operations is an effective way to provide matrix computing power. In order to support different business needs in artificial intelligence tasks, such as neural network training and reasoning, matrix computing power needs to support data formats of multiple precisions. Generally, a higher precision data format is required during training, and a lower precision data format is used during reasoning. Therefore, the basic hardware implementation of matrix operations (multiplication and addition operations) is required to support data formats of multiple precisions. In the prior art, floating-point computing units that support multiple precisions generally achieve higher multiplier utilization by adopting a low-precision combination strategy. These technical solutions only support operations in floating-point data formats, but not integer data formats, and do not support low-precision floating-point data formats such as FP8, resulting in limited application scenarios and limited computing power in scenarios where low-precision data formats can be used for operations. To solve the above problems, an embodiment of the present invention proposes a reconfigurable multi-precision dot product operation unit, Figure 4 is a schematic diagram of a reconfigurable multi-precision dot product operation unit applicable to an embodiment of the present invention, such as Figure 4 As shown, the hardware includes:

[0185] 1. Input unpacking and special value detection module

[0186] The input unpacking and special value detection module (equivalent to the input processing unit mentioned above) is used to unpack the floating point numbers or integers in the input operation array A and the input operation array B according to the data format (hereinafter referred to as DTYPE) and operation mode used in the current operation, and detect the special values ​​in the floating point numbers in the input operation array A and the input operation array B. The special values ​​include zero, infinity and non-number. The two sets of sign, exponent, mantissa and special value results are distributed to the corresponding subsequent modules.

[0187] The DTYPE supported by this dot product operation unit (equivalent to the multi-precision dot product operation hardware mentioned above) includes but is not limited to floating point FP64, FP32, TF32, FP16, BF16, FP8 (including two formats FP8_E4M3, hereinafter referred to as E4M3, and FP8_E5M2, hereinafter referred to as E5M2) and integer INT8 format.

[0188] The operation modes supported by this unit include but are not limited to FP64·FP64, FP32·FP32, TF32·TF32, FP16·FP16, BF16·BF16, E4M3·E4M3, E5M2·E5M2, E4M3·E5M2, E5M2·E4M3 and INT8·INT8 (where · represents a dot product operation, i.e. ·The data format in front indicates the format of input operation array A, ·The data format in the back indicates the format of input operation array B).

[0189] (1) Unpacking results

[0190] The input operation array A and the input operation array B have the same data bit width, and the preferred value is 512 bits. Figure 5 Schematic diagram of index and arrangement of data in an input operation array applicable to an embodiment of the present invention. When DTYPE is different, the index and arrangement of data in the input operation array are as follows: Figure 5 shown.

[0191] When DTYPE is FP64, FP32, TF32, FP16, BF16, FP8, and INT8, 2, 8, 16, 32, 32, 64, and 64 floating-point numbers or integers in input operand array A and input operand array B need to be unpacked. When DTYPE is FP64 or FP32, the floating-point numbers that need to be unpacked only occupy a part of input operand array A and input operand array B. When DTYPE is FP64, input operand arrays A and B each have 8 floating-point FP64 format numbers, but only 2 floating-point FP64 format numbers in A and B need to be unpacked. The indexes of these two floating-point numbers can be any two different indexes from 1 to 8, but the indexes used for A and B are the same. A preferred way is to use floating-point numbers with indices 1 and 5. When DTYPE is FP32, both input operands A and B have 16 floating-point FP32 format numbers, but only 8 floating-point FP32 format numbers in A and B need to be unpacked. The indexes of these two floating-point numbers can be any 8 different indexes from 1 to 16, but the indexes used for A and B are the same. A preferred method is to use floating-point numbers with indexes 1, 2, 5, 6, 9, 10, 13, and 14. When DTYPE is other formats, all floating-point numbers in the input operands A and B need to be unpacked.

[0192] Figure 6 1 is a schematic diagram of data required for unpacking different data formats applicable to an embodiment of the present invention. The data required for unpacking different DTYPEs is as follows: Figure 6 As shown in the figure, the data at positions 3, 4, 7, 8, 11, 12, 15 and 16 in FP32 and the data at positions 2, 3, 4, 6, 7 and 8 in FP64 represent data not used in the preferred embodiment.

[0193] (2) Index Arrangement

[0194] Figure 7 : is a schematic diagram of a preferred arrangement of indexes split from different data formats applicable to an embodiment of the present invention. The preferred arrangement scheme of indexes split from different DTYPEs is as follows: Figure 7As shown, the total bit width of the split exponent is 384 bits. When DTYPE is a floating point format, the exponent bit field is extracted from the floating point format according to the IEEE 754 standard. When DTYPE is FP64, two FP64 data need to be split. The exponent of each FP64 data is 11 bits, which are located in the lowest 11 bits of the two 192 bits, and the values ​​of the remaining bit fields are 0. When DTYPE is FP32, eight FP32 data need to be split. The exponent of each FP32 data is 8 bits, which are located in the lower 8 bits of the eight 48 bits equally divided from the 384 bits, and the values ​​of the remaining bit fields are 0. For other floating point formats, the arrangement principle is as follows: Figure 7 When DTYPE is INT8, the exponent value is all 0.

[0195] (3) Symbol arrangement

[0196] Figure 8 : is a schematic diagram of a floating point format symbol arrangement principle applicable to an embodiment of the present invention. The preferred arrangement scheme of the symbols split at different DTYPEs is as follows: Figure 8 As shown, the total bit width of the split symbols is 64 bits. When DTYPE is a floating point format, the sign bit field is extracted from the floating point format according to the IEEE 754 standard. When DTYPE is FP64, two FP64 data need to be split. The symbol split from each FP64 data is 1 bit, located in the lowest 1 bit of the two upper and lower 32 bits, and the values ​​of the remaining bit fields are 0. When DTYPE is FP32, eight FP32 data need to be split. The symbol split from each FP32 data is 1 bit, located in the lowest 1 bit of the eight 8 bits equally divided from the 64 bits, and the values ​​of the remaining bit fields are 0. For other floating point formats, the arrangement principle is as follows: Figure 8 When DTYPE is INT8, the symbol value is all 0.

[0197] (4) Arrangement of last digits

[0198] Fig. 9 : is a schematic diagram of a preferred arrangement of the mantissas split from different data formats applicable to an embodiment of the present invention. The preferred arrangement scheme of the mantissas split from different DTYPEs is as follows: Fig. 9As shown, the total bit width of the split mantissa is 512 bits. When DTYPE is a floating point format, the mantissa bit field is extracted from the floating point format according to the IEEE 754 standard, and the mantissa is processed according to the exponent value. The specific processing principle is: when the exponent value is not 0, 1 bit of 1 is added before the highest bit of the extracted mantissa bit field; when the exponent value is 0, the extracted mantissa bit field is shifted left by 1 bit. When DTYPE is FP64, 2 FP64 data need to be split and processed. The mantissa obtained by processing each FP64 data is 53 bits, which is located in the lowest 53 bits of the two 256 bits, and the values ​​of the remaining bit fields are 0. When DTYPE is FP32, 8 FP32 data need to be split and processed. The mantissa obtained by processing each FP32 data is 24 bits, which is located in the lower 24 bits of the eight 64 bits divided equally from 512 bits, and the values ​​of the remaining bit fields are 0. For other floating point formats, the arrangement principle is Fig. 9 When DTYPE is INT8, the mantissa value is a tight arrangement of 64 INT8 data.

[0199] The unpacking result, exponent arrangement, sign arrangement and mantissa arrangement correspond to the functions of the input unpacking subunit described above.

[0200] (5) Special value detection and result arrangement (corresponding to the function of the special value detection unit mentioned above)

[0201] When DTYPE is in E4M3 format, the logic of special value detection is: when the exponent and mantissa of the floating-point number are both 0, it is detected as zero; when the exponent and mantissa of the floating-point number are both the maximum value, it is detected as non-number. There is no infinity in the E4M3 format, so the detected infinity is always 0. When DTYPE is other floating-point formats, the logic of special value detection is: when the exponent and mantissa of the floating-point number are both 0, it is detected as zero; when the exponent of the floating-point number is the maximum value and the mantissa is 0, it is detected as infinity; when the exponent of the floating-point number is the maximum value and the mantissa is not 0, it is detected as non-number.

[0202] Fig.10 : is a schematic diagram of an arrangement principle of a floating-point format special value detection result applicable to an embodiment of the present invention. The preferred arrangement schemes of the three special value detection results are as follows when the DTYPE is different. Fig.10As shown, the total bit width of the detected special value is 64 bits. When DTYPE is FP64, 2 FP64 data need to be detected. The special value result detected by each FP64 data is 1 bit, located in the lowest 1 bit of the two upper and lower 32 bits, and the values ​​of the remaining bit fields are 0. When DTYPE is FP32, 8 FP32 data need to be detected. The special value result detected by each FP32 data is 1 bit, located in the lower 1 bit of the eight 8 bits divided equally from 64 bits, and the values ​​of the remaining bit fields are 0. For other floating point formats, the arrangement principle is as follows: Fig.10 When DTYPE is INT8, the special value result is all 0s.

[0203] 2. Symbol processing and special result judgment module

[0204] The symbol processing and special result judgment module (equivalent to the symbol processing unit and special result judgment unit mentioned above) is used to perform bitwise XOR operation on the two sets of input symbol results to obtain the sign of the product, and combine the two sets of special value results to judge whether the output has a special result. Special results include infinite results and non-number results, and the corresponding sign will be output when the result is infinite. The sign of the product will be sent to the multi-precision dot product device (equivalent to the multi-precision dot product calculation unit mentioned above) and the multi-precision adder. The special result and the infinity symbol will be sent to the result packaging module.

[0205] Fig.11 1 is a circuit diagram of a preferred symbol processing and special result judgment module applicable to an embodiment of the present invention, a circuit for preferably calculating the sign of a product and judging whether a special result appears in the output is as shown in FIG. Fig.11 As shown, it can be realized by using some basic logical operations. The input "zero A" and "zero B" in the figure represent the zero results obtained by detecting the input operation array A and the input operation array B respectively. Other representation methods such as "infinity A" and "non-number B" are parsed using the same method.

[0206] 3. Exponent processing and shift value calculation module

[0207] The exponent processing and shift value calculation module (equivalent to the exponent processing unit and shift value calculation unit mentioned above) is used to add the two sets of input exponent results element by element to obtain the exponent of each product, correct the exponent of the product according to whether the product is zero, and compare the exponents of these products to obtain the maximum value of the product exponent. Then, the exponent of each product is subtracted from the maximum value of the product exponent to obtain the exponent difference of each product. Finally, the shift value of each product is determined based on the exponent difference of these products and DTYPE. The shift values ​​of these products will be sent to the multi-precision dot product device and the FP64 product exchange and summation module. The maximum value of the product exponent is sent to the result packaging module as the exponent of the dot product result.

[0208] Fig.12 is a schematic diagram of a preferred implementation method of the exponential processing and shift value calculation module applicable to the embodiment of the present invention, such as Fig.12 As shown:

[0209] (1) A preferred implementation of the vector addition unit is as follows: the 384-bit exponent A and exponent B obtained after unpacking the input operation array A and the input operation array B are first vector-added, and when adding, the exponent A and the exponent B are each divided into 32 parts with 12 bits as the unit, and each part is summed using a 12-bit adder, and a total of 32 12-bit adders are used to complete the vector addition to obtain the initial product exponent. DTYPE is not concerned during vector addition, because carry space has been reserved for each individual exponent when the exponent is unpacked.

[0210] (2) The preferred implementation of the multi-precision correction unit is as follows: First, perform bitwise OR on the special value detection results zero A and zero B to obtain a detection result of a product of 0 when DTYPE is in floating point format, and correct the initial product index accordingly. After correction, a corrected product index is obtained. During multi-precision correction, the initial product index is corrected to 0 only when the product is detected to be 0, otherwise no correction is performed. And it is necessary to determine how many bits in the initial product index are set to 0 according to DTYPE. For example, when DTYPE is in E4M3 or E5M2 format, it is determined whether to set the corresponding 6 bits of the initial product index after 6 bits are divided into 64 parts to 0 according to each bit in the detection result of the product of 0. When DTYPE is in other floating point formats, it is determined whether to set the corresponding 12 bits of the initial product index after 12 bits are divided into 32 parts to 0 according to each even-numbered bit (bits index starts from 0) in the detection result of the product of 0.

[0211] (3) A preferred implementation of the multi-precision maximum value unit is: when DTYPE is in E4M3 or E5M2 format, the modified product exponent is divided into 64 parts in units of 6 bits, and the 64 exponents are compared to obtain the maximum value using a binary tree structure, and 6 bits of 0 are extended in the high position of the maximum value to form a 12-bit maximum value result. When DTYPE is in other floating point formats, the modified product exponent is divided into 32 parts in units of 12 bits, and the 32 exponents are compared to obtain the maximum value using a binary tree structure to obtain a 12-bit maximum value result. This maximum value is also used as the exponent of the dot product result.

[0212] (4) A preferred implementation of the extended subtraction unit is: when DTYPE is E4M3 or E5M2, the lower 6 bits of the maximum value are copied 64 times, and when DTYPE is other formats, the full 12 bits of the maximum value are copied 32 times to form a vector format maximum value, and a vector subtraction similar to the vector addition unit is used to subtract the corrected product exponent from it to obtain the product exponent difference.

[0213] (5) A preferred implementation of the multi-precision clamping unit is: according to the requirements for the precision of the dot product operation, the product exponent difference is clamped to the required range to obtain the clamped product exponent difference, thereby reducing the bit width of the subsequent shift alignment of the product, which helps to reduce the chip area. For example, when DTYPE is E4M3 or E5M2, according to the requirements for the precision of the dot product operation, the product exponent difference needs to be clamped to the range of [0, 16], then the 384-bit product exponent difference is divided into 64 parts with 6 bits, and it is determined whether the product exponent difference of each part is greater than 16. If it is greater than 16, the product exponent difference is set to 16, otherwise the value of the product exponent difference is not modified. For another example, when DTYPE is other floating-point formats, according to the accuracy requirement of the dot product operation, the product exponent difference needs to be clamped to the range of [0, 32]. The 384-bit product exponent difference is divided into 32 parts with 12 bits, and it is determined whether the product exponent difference of each part is greater than 32. If it is greater than 32, the product exponent difference is set to 32, otherwise the value of the product exponent difference is not modified.

[0214] (6) A preferred implementation of the multi-precision shift value calculation unit is as follows: in order to reduce the area, the same logic resources need to be reused in the subsequent shift alignment stage of the product, and the effective data bit width of the product is different for different DTYPEs. In order to reuse the same shift alignment logic, different DTYPEs have different corresponding fixed shift offsets. Therefore, a multi-precision shift value calculation unit is also needed to add a shift offset to the clamped exponent difference to form a shift value of the product.

[0215] 4. Multi-precision dot product module

[0216] The multi-precision dot product module (equivalent to the multi-precision dot product calculation unit mentioned above) is used to calculate the product or partial product of each mantissa according to the two sets of mantissas and the sign result of the input. If DTYPE is a floating point E4M3, E5M2, FP16 or BF16 (these formats are collectively referred to as floating point 8b / 16b) format, the variable value shift alignment of the floating point product or partial product is performed according to the shift value of the product output by the exponent processing and shift value calculation module. Otherwise, the floating point product or partial product is shifted and aligned by a fixed value according to DTYPE. After that, the aligned products or partial products are summed for the first time to reduce the number of products or partial products by half, and the root is According to DTYPE, the partial products after the first summation are shifted and aligned by fixed values. This step is mainly to align the partial products in FP32 and FP64 formats. After that, the aligned results are summed again to reduce the number of products or partial products to 1 / 4. Finally, if DTYPE is FP32, the variable value shift alignment of the floating-point product is performed according to the shift value of the product output by the exponent processing and shift value calculation module. Otherwise, no shift alignment is required. The result is the partial product when DTYPE is FP64, the aligned product when DTYPE is FP32, and the partial dot product result when DTYPE is the rest of the formats. The result is sent to the multi-precision adder module.

[0217] (1) Fig.13 is a schematic diagram of a preferred implementation of a multi-precision dot product module applicable to an embodiment of the present invention. A preferred implementation of the multi-precision dot product module is as follows Fig.13 As shown, its input is the product sign, product shift value, mantissa A and mantissa B unpacked from input operand array A and input operand array B. The output has different meanings for different DTYPEs. When DTYPE is FP64, it is the partial product, when DTYPE is FP32, it is the aligned product, and when DTYPE is other formats, it is the partial dot product result. The result is sent to the multi-precision adder module.

[0218] In a preferred solution of an embodiment of the present invention, 8 multi-precision dot product modules are included, each of which includes 4 mantissa multipliers (equivalent to the mantissa multiplication subunits mentioned above), and a total of 32 mantissa multipliers are included, which are responsible for completing 2 FP64 multiplications, 8 FP32 multiplications, 16 TF32 multiplications, 32 FP16 / BF16 multiplications, and 64 E4M3 / E5M2 / INT8 multiplications. When DTYPE is FP64 and FP32, a low-precision combination strategy is used, and multiple mantissa multipliers jointly calculate a product result. When DTYPE is E4M3, E5M2 and INT8, a high-precision segmentation strategy is used, so that one mantissa multiplier calculates two product results. When DTYPE is TF32, FP16 and BF16, one mantissa multiplier calculates one product result, but when DTYPE is TF32, only half of the mantissa multipliers are used. The technical solution proposed in the embodiment of the present invention improves the resource utilization rate and computing efficiency of the multiplier by integrating low-precision combination and high-precision segmentation strategies.

[0219] (2) Input bit width

[0220] The bit width of the two inputs of each mantissa multiplier is 16 bits. For the sake of clarity, these two inputs are referred to as OPA and OPB in the following text. In order to support high-precision segmentation strategies, when DTYPE is E4M3, E5M2 and INT8, two products are output. When DTYPE is the other formats, two partial products are output. For the sake of clarity, these two products or partial products are referred to as PRODA and PRODB in the following text. A preferred mantissa multiplier uses a booth encoding plus a tree adder structure to implement signed number multiplication. This structure uses OPB for booth encoding. The multiplier supports both 8b and 16b modes. When working in 16b mode, the function implemented by the mantissa multiplier is PRODA+PRODB<<8=OPA*OPB. At this time, PRODA and PRODB are partial products, and the bit width is 24 bits. When working in 8b mode, the function implemented by the mantissa multiplier is PRODA=OPA[7:0]*OPB[7:0] and PRODB=OPA[15:8]*OPB[15:8]. At this time, PRODA and PRODB are products, and the bit width is also 24 bits, of which the effective bit width is 16 bits, and the high 8 bits are the sign bit.

[0221] To support the low-precision combination strategy, multiple multipliers are combined as needed to calculate a product at different DTYPEs. In this application scenario, in order to reduce logic resources, the mantissa multiplier changes the effective bit width of one of the input OPAs as needed.

[0222] Fig.14Schematic diagram of the type of mantissa multiplier included in a preferred implementation method applicable to an embodiment of the present invention. The type of mantissa multiplier included in the preferred implementation method is as follows: Fig.14 As shown in the figure, the form of "12bx16b" is used to indicate that the effective bit widths of OPA and OPB are 12 bits and 16 bits respectively. In this preferred implementation, a total of three mantissa multipliers are included, and the effective bit widths of each OPA are 12 bits, 14 bits and 16 bits respectively, and the effective bit width of OPB is 16 bits. It should be noted that the effective bit width here is different from the actual bit width of the mantissa multiplier input port. The actual bit widths of the OPA ports of the above three mantissa multipliers are all 16 bits, and the effective bit width is not greater than the actual bit width. In actual operations, only the effective bit width is involved in the calculation.

[0223] When DTYPE is in FP64 format, the bit width of the mantissa of a single data after sign extension is 54 bits, which is greater than 16 bits, and the data needs to be segmented. The basis for segmentation is to minimize the effective bit width of the mantissa multiplier OPA. Fig.15 is a schematic diagram of a preferred segmentation method in which the data format is FP64 format applicable to an embodiment of the present invention, such as Fig.15 As shown, A1 in the input operation array A is divided into 4 segments A10, A11, A12 and A13 in the order of 13 bits, 13 bits, 13 bits and 15 bits from low to high. For B1 in the input operation array B, it is divided into 4 segments B10, B11, B12 and B13 in the order of 11 bits, 11 bits, 16 bits and 16 bits from low to high. For A5 and B5, the same segmentation method as A1 and B1 is used respectively.

[0224] After the segmentation is completed, the products of each segment of A and B are distributed to different mantissa multipliers. A total of 4x4x2=32 mantissa multipliers are required to complete the multiplication of 2 FP64 (A1xB1 and A5xB5 respectively).

[0225] Fig.16 is a schematic diagram of a preferred mantissa multiplier allocation scheme for a data format of FP64 format applicable to an embodiment of the present invention, such as Fig.16 As shown, 16 mantissa multipliers in 4 multi-precision dot product devices are used to complete 1 FP64 multiplication.

[0226] It should be noted that when assigning segments of A or B to the mantissa multiplier, if the bit width of the segment is lower than the requirement, it is necessary to use 0 extension in the high bit to meet the bit width requirement. When the product sign is negative, the result of booth encoding of OPB needs to be bitwise inverted to obtain the correct signed product. After calculating the partial products of each segment, these products need to be integrated to obtain the correct result. Taking A1xB1 as an example, the integration method of FP64 products is as follows:

[0227] A1×B1=A10×B10+A10×B11<<11+A10×B12<<22+A10×B13<<38+A11×B10<<13+A11×B11<<24+A11×B12<<35+A11×B13<<51+A12×B10<<26+A12×B11<<37+A12×B12<<48+A12×B13<<64+A13×B10<<39+A13×B11<<50+A13×B12<<61+A13×B13<<77

[0228] When DTYPE is in FP32 format, the bit width of the mantissa of a single data after sign extension is 25 bits, which is greater than 16 bits, and the data needs to be segmented. The basis for segmentation is to minimize the effective bit width of the mantissa multiplier OPA. Fig.17 FIG. 1 is a schematic diagram of a preferred segmentation method for a data format of FP32 format applicable to an embodiment of the present invention, such as Fig.17 As shown, A1 in the input operation array A is divided into two segments, A10 and A11, in the order of 13 bits and 12 bits from low to high. B1 and B5 in the input operation array B are divided into two segments, B10 and B11, in the order of 11 bits and 14 bits from low to high. The remaining valid data in the input operation arrays A and B use the same segmentation method as A1 and B1.

[0229] After the segmentation is completed, the products of each segment of A and B are assigned to different mantissa multipliers. A total of 4x8=32 mantissa multipliers are required to complete the multiplication of 8 FP32. Fig.18 is a schematic diagram of an allocation scheme of a preferred mantissa multiplier in a data format of FP32 format applicable to an embodiment of the present invention, such as Fig.18 As shown in the figure, 4 mantissa multipliers in a multi-precision dot product unit are used to complete 1 FP32 multiplication. After calculating the partial products of each segment, these products need to be integrated to get the correct result. Taking A1xB1 as an example, the integration method of FP32 products is as follows:

[0230] A1×B1=A10×B10+A10×B11<<11+A11×B10<<13+A11×B11<<24

[0231] When DTYPE is in TF32 format, the bit width of the mantissa of a single data after sign extension is 12 bits, which is less than 16 bits, and can be directly calculated using a single mantissa multiplier. Only 16 mantissa multipliers are needed to calculate 16 products in TF32 format, and the utilization rate is only half. In the preferred implementation, in order to reduce power consumption, only half of the mantissa multipliers are used in each multi-precision dot product. Fig.19 FIG. 1 is a schematic diagram of a preferred mantissa multiplier allocation scheme for a data format of TF32 format applicable to an embodiment of the present invention, such as Fig.19 As shown, the two mantissa multipliers on the left side of each dot product in the figure are in the closed state.

[0232] When DTYPE is in BF16 or FP16 format, the bit width of the mantissa of a single data after sign extension does not exceed 12 bits, which is also less than 16 bits. A single mantissa multiplier can be used directly for calculation. Calculating 32 products in BF16 or FP16 format requires 32 mantissa multipliers. At this time, each mantissa multiplier in all multi-precision dot product units is responsible for calculating the product of 1 data. Fig. 20 Schematic diagram of a preferred mantissa multiplier allocation scheme for a data format of BF16 or FP16 format applicable to an embodiment of the present invention. Fig. 20 shown.

[0233] In a preferred embodiment of the present invention, each multi-precision dot product device includes 4 floating-point 8 / 16b alignment units (equivalent to the floating-point alignment subunits mentioned above). When DTYPE is E4M3, E5M2, FP16, BF16 and TF32, the two products or partial products output by the mantissa multiplier are shifted and aligned according to the product shift value. When DTYPE is INT8, the product is already aligned, and there is no need to shift the two products output by the mantissa multiplier. When DTYPE is FP32 and FP64, PRODB needs to be left shifted by a fixed 8 bits.

[0234] In a preferred embodiment of the present invention, each multi-precision dot product device includes four first-level summing units, which calculate the sum of two products or partial products after shift alignment to obtain a first-level summing result.

[0235] In a preferred embodiment of the present invention, each multi-precision dot product device includes an FP32 / FP64 partial product alignment unit (equivalent to the partial product alignment subunit mentioned above). When DTYPE is in FP32 or FP64 format, each of the four first-level summation results is fixedly shifted and aligned, and the fixed shift value of each first-level summation result is given by the above two integration methods. When DTYPE is in other formats, there is no need to shift-align the first-level summation result, and four aligned first-level summation results are output.

[0236] In the preferred embodiment of the present invention, each multi-precision dot product unit includes a secondary summation unit, and the four aligned primary summation results are summed again to obtain a secondary summation result. The result is interpreted as a single product when DTYPE is FP32 format, as a partial product when DTYPE is FP64, and as a partial dot product result when DTYPE is other formats.

[0237] In the preferred embodiment of the present invention, each multi-precision dot product device includes an FP32 alignment unit. When DTYPE is FP32, the secondary summation result is shift-aligned according to the product shift value. When DTYPE is in other formats, no shift alignment is required. The result is interpreted as a single product after alignment when DTYPE is in FP32 format, as a partial product when DTYPE is FP64, and as a partial dot product result when DTYPE is in other formats.

[0238] 5.Multi-precision adder module

[0239] The multi-precision adder module (equivalent to the multi-precision addition calculation unit mentioned above) is used to sum the results of the multi-precision dot product output according to DTYPE. When DTYPE is FP64, the summation result is the product of two FP64s. When DTYPE is other data formats, the summation result is the dot product result of several inputs. The result is sent to the FP64 product exchange and summation module.

[0240] When DTYPE is FP64, the partial products output by the multi-precision dot product units 1 to 4 and the multi-precision dot product units 5 to 8 are summed to obtain two products. When DTYPE is in other formats, the 8 input products or partial dot product results are summed to obtain the dot product result. A preferred implementation is to use a multiplexer and a tree adder.

[0241] 6.FP64 product exchange and summation module

[0242] The FP64 product exchange and summation module (equivalent to the product exchange and summation unit mentioned above) is used to selectively exchange the two input FP64 products and product shift values ​​according to the product shift value output by the exponent processing and shift value calculation module, so that the two exchanged products and product shift values ​​are arranged in descending order according to the exponent. When the exponent values ​​are the same, the two can be in any order. After that, the product with a smaller exponent is variable shifted and aligned according to the shift value. Finally, the sum of the two aligned products is calculated to obtain the dot product result when DTYPE is FP64. When DTYPE is in other formats, these operations are not performed and the input is directly passed to the output. This module ensures that the output dot product result format is unified regardless of the format of DTYPE. The result is sent to the normalization and rounding module.

[0243] When DTYPE is in FP64 format, the exponent processing and shift value calculation module will output the shift values ​​of two FP64 products, respectively referred to as the first and second product shift values. When the first product shift value is 0, there is no need to exchange the two FP64 products and shift values. Otherwise, the two products and shift values ​​need to be exchanged. Preferably, the exchange logic of the products and shift values ​​can be implemented using a multiplexer. After the exchange, the first product and product shift value are always the product with a larger exponent value and its corresponding shift value (fixed to 0), and the second product and product shift value are always the product with a smaller exponent value and its corresponding shift value (variable value). When DTYPE is in other formats, no product exchange will be performed.

[0244] When DTYPE is in FP64 format, after the product and product shift value are exchanged, the second product is variable shift aligned according to the second product shift value, and then the first product and the shifted second product are summed to obtain the FP64 dot product result. When DTYPE is in other formats, shift alignment and summing are not performed. This module ensures that the format of the output dot product result is unified regardless of the format of DTYPE.

[0245] 7. Normalization and rounding modules

[0246] The normalization and rounding module (equivalent to the normalization and rounding unit mentioned above) is used to normalize and round the input dot product result to obtain the dot product mantissa result and sign that meet the FP64 format requirements.

[0247] The module first detects the number of header symbols of the input dot product result, and performs the initial normalization of the dot product result in combination with the exponent value, and then rounds the normalized result. In the preferred implementation, it supports rounding to 0, rounding to positive infinity, rounding to negative infinity, and rounding to the nearest even number. After rounding, the absolute value of the rounded result is taken, and normalized again to obtain the dot product mantissa result and sign that meet the FP64 format requirements. During the normalization process, the exponent value also needs to be modified accordingly according to the normalization result.

[0248] 8.Result Packaging Module

[0249] The result packing module (equivalent to the result packing unit mentioned above) packs the results according to the input special result, sign, exponent and dot product mantissa to form a dot product result in FP64 format.

[0250] When the sign processing and special result judgment module detects a special result such as infinity or non-number, the packing module directly outputs the corresponding special result. If no special result is detected, the result is packed according to the sign, exponent and dot product mantissa. During the packing process, it is determined whether the exponent exceeds the representation range of the FP64 format. Numbers that exceed the representation range will be rounded to infinity or the maximum value according to the rounding mode. If they do not exceed the representation range, the normal value will be output.

[0251] A reconfigurable multi-precision dot product operation unit proposed in an embodiment of the present invention can be automatically reconstructed into a dot product operation unit suitable for the data format according to the set data format, and this reconstruction is dynamically configurable on the chip. Regardless of the DTYPE used, the output dot product result is in FP64 format, which ensures the high precision of the dot product result and reduces the subsequent cumulative error. A rich range of DTYPEs are supported, including but not limited to FP64, FP32, TF32, FP16, BF16, E4M3, E5M2 and INT8 formats. The supported DTYPEs include high-precision data formats such as FP64 and FP32 for supercomputing, as well as low-precision data formats such as BF16 and FP8 for artificial intelligence, so it is particularly suitable for supercomputing and artificial intelligence application scenarios. The exponential arrangement scheme output by the input unpacking module can use the same adder to achieve exponential summation for any DTYPE without any additional processing. The low-precision combination strategy and high-precision splitting strategy of the mantissa multipliers in the multi-precision dot product device use multiple mantissa multipliers to combine and calculate a high-precision result in FP64 and FP32 formats. In E4M2, E5M3 and INT8 formats, one mantissa multiplier can be split into two multipliers, which improves the resource sharing rate.

[0252] In addition, a reconfigurable multi-precision dot product operation unit proposed in an embodiment of the present invention can change the supported DTYPE to support more or fewer data formats as needed; the supported rounding mode can be more or less as needed; the output data format is not limited to the FP64 format, and can be changed to other data formats as needed. When using a low-precision combination strategy, the input bit width of each mantissa multiplier and the allocation method of input data can be changed according to usage needs. In a reconfigurable multi-precision dot product operation unit proposed in an embodiment of the present invention, any number of pipeline registers can be inserted to optimize the operation delay and increase the system operating frequency. In a reconfigurable multi-precision dot product operation unit proposed in an embodiment of the present invention, the data bit width in the input operation array A and the input operation array B is not limited to 512 bits, and can be modified to other bit widths according to the supported DTYPE. Correspondingly, the number of multi-precision dot product devices is not limited to 8 and can be modified according to usage needs.

[0253] The embodiment of the present invention proposes a reconfigurable multi-precision dot product operation unit, in which calculation units such as exponential summation, comparison, mantissa multiplication, alignment shift and summation can be reused under various DTYPEs, reducing the design area and power consumption, and having a high hardware resource utilization rate; it supports a wide range of data formats, not only supporting low-precision floating-point data formats, but also supporting integer data formats, with a wider range of application scenarios, which can meet the application requirements of supercomputing and artificial intelligence application scenarios; a single operation unit can complete the dot product operation of a large amount of data at a time, improving computing power and efficiency; the calculation accuracy is adjustable, and the calculation accuracy of the dot product operation can be conveniently changed by adjusting the upper limit of the shift value during product shift alignment, thereby meeting the needs of different applications.

[0254] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0255] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-precision dot product computing hardware, characterized in that: include: An input processing unit, a dot product operator, and an output processing unit; An input processing unit is used to unpack the two input operation arrays according to the pre-configured input data format, and obtain the sign distribution pattern, exponent distribution pattern and mantissa distribution pattern corresponding to each operation array according to the unpacking result and the input data format; obtain the special value judgment result of the dot product result, and calculate the sign of the product of each paired operand in the two operation arrays; A dot product operator is used to calculate the exponent of the dot product result and the shift value of the product of each pair of operands in the two operation arrays according to the exponential distribution pattern of each operation array; calculate each product or partial product between each pair of operands in the two operation arrays according to the sign distribution pattern, the mantissa distribution pattern of each operation array and the sign of the product of each pair of operands; and perform at least one shift alignment and at least one summation process on each product or partial product according to the input data format and / or the shift value of the product of each pair of operands to obtain multiple dot product operation intermediate results; A sum operation is performed on the intermediate results of multiple dot product operations to obtain a dot product result; the dot product result includes a dot product mantissa and a dot product sign; The output processing unit is used to round and normalize the dot product mantissa according to the pre-configured standardized output format, rounding type and exponent of the dot product result; obtain the normalized dot product mantissa, normalized dot product sign and normalized dot product exponent that meet the standardized output format as the normalized processing result; and package and output the normalized processing result according to the special value judgment result of the dot product result.

2. The multi-precision dot product computing hardware according to claim 1, characterized in that: The input processing unit specifically includes an input unpacking subunit, a special value detection unit, a symbol processing unit, and a special result judgment unit, wherein: An input unpacking subunit is used to determine the number of unpacking and the unpacking position corresponding to each operation array according to the input data format after obtaining multiple data packets corresponding to each operation array; according to the number of unpacking and the unpacking position, the sign, exponent and mantissa of each operand in each operation array are separated in each data packet; according to the data arrangement method matching the input data format, the separated sign, exponent and mantissa are filled into the matching pattern template respectively, so as to obtain the symbol distribution pattern, exponent distribution pattern and mantissa distribution pattern corresponding to each operation array; A special value detection unit is used to determine the zero value detection result, infinite value detection result, and non-number detection result of each operand in each operation array according to the input data format and the exponent and mantissa of each operand in each operation array; fill each zero value detection result, infinite value detection result, and non-number detection result into a matching pattern template according to a data arrangement method matching the input data format, and obtain a zero value distribution pattern, an infinite value distribution pattern, and a non-number distribution pattern corresponding to each operation array; A symbol processing unit, used for performing a bitwise XOR operation on the symbol distribution pattern corresponding to each operation array, to obtain the sign of the product of each pair of operands in the two operation arrays; The special result judgment unit is used to perform logical judgment on the zero value distribution pattern, infinite value distribution pattern and non-number distribution pattern of each operation array through a pre-connected logic gate array, and obtain the zero value judgment result, infinite value judgment result and non-number judgment result corresponding to the dot product result respectively.

3. The multi-precision dot product computing hardware according to claim 2, characterized in that: The input unpacking subunit is further used for: After filling the split current filling mantissa into the matching mantissa bit field in the matching style template according to the data arrangement method matching the input data format, if the exponent corresponding to the current filling mantissa is not 0, the previous bit of the highest bit of the mantissa bit field is set to 1; if the exponent corresponding to the current filling mantissa is 0, the mantissa bit field is shifted left by 1 bit to obtain a mantissa distribution pattern.

4. The multi-precision dot product computing hardware according to claim 2, characterized in that: The dot product operator specifically includes: an exponent processing unit, a shift value calculation unit, at least two multi-precision dot product calculation units and a multi-precision addition calculation unit, wherein: An exponential processing unit, used to calculate the exponent of a dot product result obtained after performing dot product calculation on two operation arrays according to the exponential distribution pattern and the zero value distribution pattern of each operation array; A shift value calculation unit, for calculating a shift value of a product of each pair of operands in two operand arrays according to an exponent of a dot product result; Each multi-precision dot product calculation unit is used to calculate the product or partial product of at least one group of paired operands according to the sign distribution pattern, the mantissa distribution pattern of each operand array and the sign of the product of each paired operand; and perform at least one shift alignment and at least one summation process on each product or partial product according to the input data format and / or the shift value of the product of each paired operand, to obtain an intermediate result of the dot product operation, wherein the intermediate result of the dot product operation includes the product of a group of paired operands, the partial product of a group of paired operands, or the sum of the products of two groups of paired operands; The multi-precision addition calculation unit is used to perform a sum operation on the intermediate results of dot product calculations output by multiple multi-precision dot product calculation units to obtain a dot product result.

5. The multi-precision dot product computing hardware according to claim 4, characterized in that: The index processing unit is specifically used for: Perform vector addition of the exponential distribution patterns of each operation array to obtain an initial product exponential pattern; Performing a bitwise OR operation on the zero value distribution pattern of each operation array, and obtaining a test result of whether the product of each pair of operands in the two operation arrays is 0; According to the input data format and the detection result of whether the product of each pairing operand is 0, the initial product exponential pattern is exponentially corrected to obtain a corrected product exponential pattern; According to the input data format and the modified product index pattern, the maximum value of the product index among the product indexes corresponding to the products of each pair of operands in the two operation arrays is compared; The maximum value of the product exponents obtained by comparison is determined as the exponent of the dot product result.

6. The dot product computing hardware according to claim 5, characterized in that: The shift value calculation unit is specifically used for: According to the input data format, the exponent of the dot product result is expanded to obtain a vector format expanded exponent; Subtracting the corrected product index pattern from the vector format expanded index to obtain a product index difference pattern; According to the input data format, the product exponent differences corresponding to the products of each pairing operand in the product exponent difference pattern are clamped to obtain a clamped product exponent difference pattern; According to the input data format, a fixed shift offset is determined, and the fixed shift offset is added to the clamped product exponent difference pattern to obtain a shift value of the product of each paired operand in two operation arrays.

7. The dot product operation hardware according to claim 4, characterized in that: The multi-precision dot product calculation unit specifically includes a plurality of mantissa multiplication subunits, a floating point alignment subunit and a first-level summation subunit corresponding to each mantissa multiplication subunit, a partial product alignment subunit, a second-level summation subunit and a standardized output alignment subunit, wherein: A mantissa multiplication subunit, configured to perform a signed multiplication calculation according to two allocated input data and a sign of a product of a paired operand matching the input data, to obtain two products or two partial products; According to different input data formats, a low-precision combination strategy, a high-precision segmentation strategy, or no strategy is selected, and input data allocated to each mantissa multiplication subunit is determined based on a sign distribution pattern and a mantissa distribution pattern of each operand array, so that a plurality of mantissa multiplication subunits jointly calculate a product result of a group of paired operands, a mantissa multiplication subunit independently calculates two product results of two groups of paired operands, or a mantissa multiplication subunit independently calculates one product result of a group of paired operands; A floating point alignment subunit, configured to perform or not perform shift alignment processing on two products or two partial products output by the matching mantissa multiplication subunit according to an input data format and a shift value of the product of each paired operand, so as to obtain two aligned products or two partial products; A first-level summing subunit is used to sum the two aligned products or two partial products output by the matching floating-point alignment subunit to obtain a first-level summing result; The partial product alignment subunit is used to select whether to perform shift alignment processing on each first-level summation result or not according to the input data format, so as to obtain a plurality of aligned first-level summation results; The secondary summation subunit is used to perform a secondary summation process on the multiple primary summation results input by the partial product alignment subunit to obtain a secondary summation result; The standardized output alignment subunit is used to select whether to perform shift alignment processing on the secondary summation result or not according to the input data format to obtain the intermediate result of the dot product operation.

8. The multi-precision dot product computing hardware according to claim 4, characterized in that: The dot product operator specifically includes a product exchange summation unit; The multiple precision addition calculation unit is specifically used to: when it is determined that the input data format belongs to the first type of data format, sum the output results of each multiple precision dot product calculation unit to obtain two products, and transmit the two products to the product exchange summation unit; And, when it is determined that the input data format belongs to the second type of data format, summing the output results of each multi-precision dot product calculation unit to obtain a dot product result, and directly transmitting the dot product result to the product exchange summation unit; A product exchange and summation unit is used for, when it is determined that the input data format belongs to the first type of data format, selectively exchanging the two products according to the shift values ​​of the products of the two products and the paired operands in the two operation arrays received, shifting and aligning the latter product according to the shift value of the matching product, and then calculating the sum of the two products as a dot product result and directly transmitting it to the output processing unit; And, when it is determined that the input data format belongs to the second type of data format, no exchange, shift alignment and summation are performed, and the received dot product result is directly transmitted to the output processing unit.

9. The dot product operation hardware according to claim 1, characterized in that: The output processing unit specifically includes a normalization and rounding unit, and a result packing unit, wherein: A normalization and rounding unit, used for detecting the number of sign bits contained in the input dot product result, and performing initial normalization on the dot product result according to the number of sign bits and the exponent of the input dot product result; performing rounding processing on the dot product result after the initial normalization according to the rounding type to obtain a rounded result; taking an absolute value of the rounded result, and normalizing it again to obtain a normalized dot product mantissa, a normalized dot product sign, and a normalized dot product exponent that meet the standardized output format as a normalization processing result; A result packing unit is used to pack and output the special value when it is determined that the dot product result is a special value according to the special value judgment result of the dot product result; when it is determined that the dot product result is not a special value according to the special value judgment result of the dot product result, pack the normalized dot product mantissa, the normalized dot product sign and the normalized dot product exponent to obtain and output the packing result that satisfies the data representation range of the standardized output format.

10. The dot product computing hardware according to any one of claims 1 to 9, characterized in that: The dot product operation hardware can perform on-chip reconstruction of configuration information in real time according to configuration information of input data format, standardized output format and rounding type matching the current computing scenario to meet the computing requirements of the current computing scenario.

Images