Single-precision floating point matrix multiplication calculation unit, splitting mapping method and accelerator

Abstract

The invention provides a single-precision floating-point matrix multiplication calculation unit, and the unit comprises a preprocessing module which is used for reading single-precision floating-point data, and dividing the mantissa field part of each matrix into a plurality of groups of integer matrixes with low bit width; the linear evaluation arithmetic logic module is used for executing linear combination operation on the multiple groups of low-bit-width integer matrixes to generate multiple groups of point value matrixes; the low-bit-width integer calculation array is used for executing low-bit-width integer matrix multiplication operation on the multiple groups of point value matrixes to obtain multiple groups of point value product matrixes; the interpolation merging module is used for executing interpolation operation and weighted merging on the multiple groups of point value product matrixes to generate a mantissa field product result; and the format conversion module is used for combining the mantissa field product result with the exponential field part of each matrix to generate a single-precision floating-point number matrix multiplication result. According to the invention, the overall throughput and energy efficiency are improved, and the mapping calculation and system overhead are reduced.

Description

Technical Field

[0001] This invention relates to the field of hardware acceleration technology, and in particular to a single-precision floating-point matrix multiplication calculation unit, a split mapping method, and an accelerator. Background Technology

[0002] With the development of workloads such as deep learning and scientific computing, single-precision floating-point (FP32) matrix multiplication remains a fundamental operator in many critical scenarios, directly impacting model training quality and the reliability of scientific computing results. However, existing AI accelerators typically allocate the majority of their area and energy budget to low-bit-width matrix computation arrays such as INT8 (8-bit signed integer data type). FP32 often relies on small, independent computation paths. When workloads must be executed in FP32, it is often difficult to fully utilize the high throughput and energy efficiency resources of low-bit-width arrays, resulting in limited overall throughput and decreased energy efficiency. Furthermore, to reuse low-bit-width arrays, a naive mantissa fragmentation method is often used, splitting FP32 operations into combinations of several low-bit-width operations. However, the mapping process requires too many sub-operations or introduces a large amount of intermediate data handling and control overhead, which offsets the benefits of array reuse, making the end-to-end performance and energy efficiency improvements insignificant. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention proposes a single-precision floating-point matrix multiplication computation unit, a split mapping method, and an accelerator. Based on Toom-Cook-3 (TC-3) matrix-level split mapping, it improves the reuse efficiency of low-bit-width arrays, enabling FP32 matrix multiplication to make fuller use of existing low-bit-width array resources, thereby improving overall throughput and energy efficiency.

[0004] This invention provides a single-precision floating-point matrix multiplication calculation unit, comprising:

[0005] The preprocessing module is used to read single-precision floating-point data and split the mantissa of each matrix into multiple low-bit-width integer matrices.

[0006] A linear evaluation arithmetic logic module, connected to the preprocessing module, is used to perform linear combination operations on the multiple sets of low-bit-width integer matrices to generate multiple sets of point-value matrices.

[0007] A low-bit-width integer computation array is connected to the linear evaluation arithmetic logic module and is used to perform low-bit-width integer matrix multiplication operations on the multiple sets of point-value matrices to obtain multiple sets of point-value product matrices.

[0008] The interpolation and merging module is connected to the low-bit-width integer calculation array and is used to perform interpolation operations and weighted merging on the multiple sets of point-value product matrices to generate the mantissa field product result.

[0009] The format conversion module, connected to the interpolation merging module, is used to combine the mantissa field product result with the exponent field part of each matrix to generate a single-precision floating-point matrix multiplication result for storage.

[0010] In one embodiment of the present invention, the preprocessing module includes a mantissa splitting module and a radix configuration register, wherein the radix configuration register pre-stores 2 0 2 -8 2 -16 The mantissa splitting module splits the mantissa field of each matrix data in the single-precision floating-point data into three groups of 8-bit integer matrices based on the three splitting radixes, resulting in three groups of low-bit-width integer matrices.

[0011] In one embodiment of the present invention, the linear evaluation arithmetic logic module includes multiple sets of parallel adders / subtractors and shifters. It performs linear combination on three sets of low-bit-width integer matrices through a preset evaluation point set {0,1,-1,2,∞} to generate five sets of point value matrices corresponding to the evaluation point set. The linear combination is implemented only through addition, subtraction and shift operations.

[0012] In one embodiment of the present invention, the low-bit-width integer computing array is an accelerator native 8-bit integer matrix multiplication array;

[0013] The five sets of point-value matrices are paired up and subjected to low-bit-width integer matrix multiplication to obtain the corresponding five sets of point-value product matrices.

[0014] In one embodiment of the present invention, the interpolation merging module includes a coefficient register, an interpolation module, and a weighted addition tree. The coefficient register pre-stores fixed coefficients for interpolation operations and predetermined base weights. The interpolation module performs interpolation operations on five sets of point value product matrices to generate five sets of interpolation component matrices. The weighted addition tree is shifted and summed according to the predetermined base weights to obtain the mantissa field product result.

[0015] In one embodiment of the present invention, it further includes: a global control module with built-in pipeline scheduling logic, which synchronizes the clock beats of each module through a control bus: wherein, while the preprocessing module splits the N+1th matrix, the low-bit-width integer calculation array performs the multiplication operation of the Nth matrix, and the interpolation merging module processes the product result of the N-1th matrix.

[0016] Another aspect of the present invention provides an AI accelerator, comprising a computing unit, a multi-level cache, and a DMA controller, wherein the computing unit comprises:

[0017] The preprocessing module is used to read single-precision floating-point data and split the mantissa of each matrix into multiple low-bit-width integer matrices.

[0018] A linear evaluation arithmetic logic module, connected to the preprocessing module, is used to perform linear combination operations on the multiple sets of low-bit-width integer matrices to generate multiple sets of point-value matrices.

[0019] A low-bit-width integer computation array is connected to the linear evaluation arithmetic logic module and is used to perform low-bit-width integer matrix multiplication operations on the multiple sets of point-value matrices to obtain multiple sets of point-value product matrices.

[0020] The interpolation and merging module is connected to the low-bit-width integer calculation array and is used to perform interpolation operations and weighted merging on the multiple sets of point-value product matrices to generate the mantissa field product result.

[0021] The format conversion module, connected to the interpolation merging module, is used to combine the mantissa field product result with the exponent field part of each matrix to generate a single-precision floating-point matrix multiplication result and transmit it to the multi-level cache.

[0022] The multi-level cache stores single-precision floating-point data and the results of single-precision floating-point matrix multiplication.

[0023] Another aspect of the present invention provides a single-precision floating-point matrix multiplication splitting mapping method, executed on an accelerator, comprising:

[0024] The mantissa portion of each matrix of the input single-precision floating-point data is split into multiple low-bit-width integer matrices.

[0025] Perform a fixed linear combination operation on the multiple sets of low-width integer matrices to generate multiple sets of point value matrices;

[0026] Perform low-bit-width integer matrix multiplication on the multiple sets of point-value matrices to obtain multiple sets of point-value product matrices;

[0027] Interpolation and weighted merging are performed on the multiple sets of point-value product matrices to obtain the mantissa field product result of single-precision floating-point matrix multiplication;

[0028] The mantissa product result is combined with the exponent field of the single-precision floating-point number to output the final single-precision floating-point number.

[0029] Precision floating-point matrix multiplication results.

[0030] In one embodiment of the present invention, the step of performing interpolation and weighted merging on the multiple sets of point-value product matrices to obtain the mantissa field product result further includes:

[0031] Interpolation operations are performed on the multiple sets of point value product matrices to obtain multiple sets of interpolation component matrices;

[0032] The multiple interpolation component matrices are weighted and merged to obtain the mantissa product of single-precision floating-point matrix multiplication.

[0033] In one embodiment of the present invention, the mantissa field of each matrix data in the single-precision floating-point data is split into three groups of 8-bit integer matrices according to a preset splitting radix, resulting in three groups of low-bit-width integer matrices; wherein the splitting radix is ​​2... 0 2 -8 2 -16 .

[0034] In one embodiment of the present invention, a linear combination is performed on three sets of low-width integer matrices using a preset evaluation point set {0,1,-1,2,∞} to generate five sets of point value matrices corresponding to the evaluation point set. The linear combination is achieved only through addition, subtraction and shift operations. The multiple sets of point value matrices are five sets.

[0035] In one embodiment of the present invention, the five sets of point value matrices are paired up and low-bit-width integer matrix multiplication is performed on each pair to obtain the corresponding five sets of point value product matrices.

[0036] As can be seen from the above solutions, the advantages of the present invention are:

[0037] This invention employs a matrix-level splitting mapping based on TC-3, mapping FP32 matrix multiplication to fewer low-bit-width integer matrix multiplications through a fixed evaluation / interpolation linear transformation. This improves the reuse efficiency of existing low-bit-width accelerator arrays while maintaining FP32 semantics and accuracy requirements, allowing FP32 matrix multiplication to make fuller use of existing low-bit-width array resources, thereby improving overall throughput and energy efficiency. Simultaneously, it reduces mapping computation and system overhead, decreases the number of sub-operations and array calls, and reduces scheduling switching and intermediate data transfer overhead. Attached Figure Description

[0038] Figure 1 This diagram illustrates the structure of a single-precision floating-point matrix multiplication calculation unit according to an embodiment of the present invention.

[0039] Figure 2 This diagram illustrates the INT8 polynomial representation of FP32 mantissa fragmentation.

[0040] Figure 3 A schematic diagram of the TC-3 calculation process is shown;

[0041] Figure 4 A schematic diagram of the overall process of a single-precision floating-point matrix multiplication splitting mapping method provided by an embodiment of the present invention is shown;

[0042] Figure 5 It shows Figure 4 A schematic diagram of the specific process for step S4.

[0043] The attached figures are labeled as follows:

[0044] 1: Computational unit;

[0045] 10: Preprocessing module;

[0046] 11: Linear evaluation arithmetic logic module;

[0047] 12: Low-bit-width integer computation array;

[0048] 13: Interpolation merging module;

[0049] 14: Format conversion module;

[0050] 2: Multi-level caching;

[0051] 3: DMA controller. Detailed Implementation

[0052] It should be noted that, in this invention, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0053] In the absence of further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0054] As mentioned earlier, existing AI accelerators such as GPUs, DPUs, IPUs, and TPUs typically allocate their majority of area and energy budget to low-bit-width matrix computing arrays like INT8. FP32 often relies on smaller, independent computing paths, leading to performance and energy efficiency limitations and difficulties in fully reusing hardware resources in tasks requiring FP32. However, FP32 remains irreplaceable in applications: for example, it is generally more robust in tasks involving long-chain accumulation, error sensitivity, or high reproducibility requirements. This stability reduces the risk of accuracy degradation due to error accumulation, thereby improving training convergence quality and the reliability of scientific computing results. FP32 has a larger dynamic range and more mature normalization / rounding semantics, making it less prone to overflow, underflow, or numerical anomalies when dealing with data distributions of different scales. This controllability reduces the additional calibration and compensation overhead introduced for "fallback numerical problems," thereby lowering system parameter tuning costs and improving deployment robustness. Even though inference can be partially quantized, FP32 remains the default or necessary choice in training, optimizer updates, partial normalization / scaling operations, and some high-precision scientific computing kernels. This necessity means that the performance of the FP32 path directly determines the end-to-end throughput and energy efficiency limits, thus becoming a key bottleneck for system optimization. To address this, this invention considers efficiently mapping FP32 matrix multiplication to low-bit-width arrays for execution, which can significantly improve accelerator resource utilization and overall energy efficiency without sacrificing precision semantics.

[0055] Specifically, refer to Figure 1 As shown, Figure 1 A schematic diagram of the structure of a single-precision floating-point matrix multiplication calculation unit provided in an embodiment of the present invention is shown.

[0056] A single-precision floating-point matrix multiplication calculation unit 1 includes:

[0057] The preprocessing module 10 is used to read single-precision floating-point data and split the mantissa of each matrix into multiple low-bit-width integer matrices.

[0058] The linear evaluation arithmetic logic module 11, connected to the preprocessing module 10, is used to perform linear combination operations on the multiple sets of low-bit-width integer matrices to generate multiple sets of point value matrices.

[0059] The low-bit-width integer calculation array 12 is connected to the linear evaluation arithmetic logic module 11 and is used to perform low-bit-width integer matrix multiplication on the multiple sets of point-value matrices to obtain multiple sets of point-value product matrices.

[0060] The interpolation and merging module 13 is connected to the low-bit-width integer calculation array 12 and is used to perform interpolation operations and weighted merging on the multiple sets of point value product matrices to generate the mantissa field product result.

[0061] The format conversion module 14, connected to the interpolation merging module 13, is used to combine the mantissa field product result with the exponent field part of each matrix to generate a single-precision floating-point matrix multiplication result and store it in the on-chip storage unit.

[0062] In this embodiment, the computing unit of the existing AI accelerator hardware structure is improved by decomposing FP32 matrix multiplication into a multi-stage parallel task adapted to low-bit-width arrays. This allows the accelerator's existing low-bit-width arrays to handle FP32 computing requirements without additional modifications, maximizing the reuse of existing low-bit-width computing resources at the hardware level and achieving a dual improvement in throughput and energy efficiency. Specifically, the preprocessing module splits the mantissa field of the FP32 matrix into multiple sets of low-bit-width integer matrices, transforming the task that originally relied on high-precision floating-point operations into parallel integer operations that can be directly processed by the low-bit-width array. Then, through the linear combination operation of the linear evaluation arithmetic logic module, the multiplication operation of the low-bit-width integer computing array, and the interpolation and weighted merging of the interpolation and merging module, the mantissa field product result is obtained, which is then combined with the original exponent field part to obtain the single-precision floating-point matrix multiplication result. This invention improves the reuse efficiency of low-bit-width arrays, enabling FP32 matrix multiplication to make fuller use of existing low-bit-width array resources, thereby improving overall throughput and energy efficiency. Furthermore, it reduces mapping computation and system overhead, decreases the number of sub-operations and array calls, and reduces scheduling switching and intermediate data transfer overhead.

[0063] In one embodiment, the preprocessing module includes a mantissa splitting module and a radix configuration register, wherein the radix configuration register pre-stores 2 0 2 -8 2 -16 The mantissa splitting module splits the mantissa field of each matrix data in the single-precision floating-point data into three groups of 8-bit integer matrices based on the three splitting radixes, resulting in three groups of low-bit-width integer matrices.

[0064] In practical implementation, when calculating FP32 matrix multiplication, for the input matrices A and B, the mantissa of each matrix can be treated as a polynomial composed of INT8 values, and then weighted. The splitting process is as follows: Figure 2 As shown, Figure 2 This diagram illustrates the FP32 mantissa fragmented INT8 polynomial representation. Matrices A and B are decomposed into three low-bit-width matrices A0, A1, A2 and B0, B1, B2, respectively, with a pre-defined base of 2. 0 2 -8 2 -16This transforms the original FP32 mantissa multiplication into an INT8 polynomial multiplication problem. Furthermore, this splitting allows the original FP32 matrix multiplication to be mapped onto low-bit multiplication arrays such as INT8, leveraging multiplexing to improve the computational efficiency of low-bit arrays, thereby enhancing overall throughput and energy efficiency.

[0065] Then, in one embodiment, a matrix-level splitting mapping method based on Toom-Cook-3 (TC-3) is further implemented. The essence of Toom-Cook-3 is to split large integers into low-order polynomials, reduce the number of multiplications by selecting specific points for evaluation, and then reassemble the results through interpolation. When applying at the matrix level, we treat the matrix as a "large integer," divide it into submatrices (corresponding to segments of the large integer), and then apply the Toom-Cook-3 "split-evaluate-dot product-interpolate-reassemble" process to these submatrix blocks.

[0066] In one embodiment, the linear evaluation arithmetic logic module includes multiple sets of parallel adders / subtractors and shifters. It performs linear combinations on three sets of low-bit-width integer matrices using a preset evaluation point set {0,1,-1,2,∞} to generate five sets of point value matrices corresponding to the evaluation point set. The linear combination is implemented only through addition, subtraction, and shift operations.

[0067] Figure 3 This is a schematic diagram of the TC-3 calculation process. In the specific implementation, for the INT8 matrix slices A0, A1, A2 and B0, B1, B2, the original mantissa polynomials A(x) and B(x) can be evaluated on a set of preset linear evaluation points {0,1,-1,2,∞}, resulting in five sets of point-value matrices A(0) ~ A(∞) and B(0) ~ B(∞). These can all be represented as fixed linear combinations of the original matrix slices (A0, A1, A2, B0, B1, B2). This part of the process corresponds to... Figure 3 The “TC-3 Evaluation” section shows that this process only involves addition / subtraction and constant multiples (which can be achieved by shifting), so the execution flow is fixed and highly regular, and the implementation process is low-cost and efficient.

[0068] Furthermore, in one embodiment, the low-bit-width integer computation array is the accelerator's native 8-bit integer matrix multiplication array. The low-bit-width integer computation array further pairs the five point-value matrices A(0) ~ A(∞) and B(0) ~ B(∞) and performs low-bit-width integer matrix multiplication on each pair, resulting in five corresponding point-value product matrices M(0), M(1), M(-1), M(2), and M(∞). Clearly, these five matrix-level element-wise multiplications correspond to the dot product characteristic of TC-3, transforming the original 9 partial products (3*3) required in the naive trinomial polynomial piecewise expansion into only 5 products, significantly reducing the number of low-precision matrix multiplications and related execution, scheduling, and merging overheads. This process corresponds to... Figure 3 The content shown in the "Low-precision dot product" section.

[0069] In one embodiment, the interpolation merging module includes a coefficient register, an interpolation module, and a weighted addition tree. The coefficient register pre-stores fixed coefficients for interpolation operations and predetermined base weights. The interpolation module performs interpolation operations on five sets of point value product matrices to generate five sets of interpolation component matrices. The weighted addition tree is shifted and summed according to the predetermined base weights to obtain the mantissa field product result.

[0070] In the specific implementation, interpolation is performed on the five sets of point-value product matrices M(0), M(1), M(-1), M(2), and M(∞) to obtain five interpolation component matrices of the polynomial product C(x), which are denoted as I(0), I(1), I(2), I(3), and I(4), respectively. This interpolation can be expressed as the result of a linear combination of fixed coefficients of the five product matrices. Therefore, the calculation process of this interpolation component matrix can be implemented using only addition, subtraction, shifting, and a small number of constant coefficient processing logics, which has a fixed and predefined simple and efficient calculation structure. After obtaining the interpolation component matrices I(0) ~ I(4), TC-3 performs weighted merging according to a predetermined base weight to obtain the final multiplication result matrix. The base weight can be determined by the power relationship of the base, for example, 2 0 2 -8 2 -16 2 -24 2 -32 The interpolated component matrices together form the coefficients of C(x) = A(x)B(x). The final result can be obtained by weighting these coefficients according to a predetermined base. The above weighted merging can be represented by regular shift and summation operations, thereby ensuring the determinism and feasibility of the recovery process. This process corresponds to... Figure 3 The content shown in the "Interpolation" section.

[0071] Finally, the format conversion module 14 combines the mantissa field product result with the exponent field part of each matrix to generate a single-precision floating-point matrix multiplication result.

[0072] Furthermore, in one embodiment, this computing unit further includes: a scheduling module responsible for globally coordinating the operations of each module, with built-in pipeline scheduling logic, synchronizing the clock cycles of each module through a control bus to eliminate operational delay differences between units, and achieving bubble-free end-to-end pipeline operation. For example, while the preprocessing module splits the N+1th matrix, the low-bit-width integer computing array performs the multiplication operation of the Nth matrix, and the interpolation merging module processes the product result of the N-1th matrix.

[0073] Furthermore, in one embodiment of the present invention, an AI accelerator is also provided, which includes a computing unit, a multi-level cache, etc., wherein the multi-level cache is connected to the computing unit, and:

[0074] The computing unit 1 further includes:

[0075] The preprocessing module 10 is used to read single-precision floating-point data and split the mantissa of each matrix into multiple low-bit-width integer matrices.

[0076] The linear evaluation arithmetic logic module 11, connected to the preprocessing module 10, is used to perform linear combination operations on the multiple sets of low-bit-width integer matrices to generate multiple sets of point value matrices.

[0077] The low-bit-width integer calculation array 12 is connected to the linear evaluation arithmetic logic module 11 and is used to perform low-bit-width integer matrix multiplication on the multiple sets of point-value matrices to obtain multiple sets of point-value product matrices.

[0078] The interpolation and merging module 13 is connected to the low-bit-width integer calculation array 12 and is used to perform interpolation operations and weighted merging on the multiple sets of point value product matrices to generate the mantissa field product result.

[0079] The format conversion module 14, connected to the interpolation merging module 13, is used to combine the mantissa field product result with the exponent field part of each matrix to generate a single-precision floating-point matrix multiplication result and store it in the multi-level cache.

[0080] The multi-level cache is connected to the computing unit and stores single-precision floating-point data, single-precision floating-point matrix multiplication results, etc.

[0081] In one embodiment, a single-precision floating-point matrix multiplication splitting mapping method is further provided. This embodiment is a method embodiment corresponding to the above embodiment, such as... Figure 4 As shown, Figure 4This diagram illustrates the overall flow of a single-precision floating-point matrix multiplication splitting mapping method according to an embodiment of the present invention. This method can be implemented in conjunction with the embodiments described above. The relevant technical details mentioned in the above embodiments remain valid in this method embodiment, and will not be repeated here to avoid repetition.

[0082] A single-precision floating-point matrix multiplication splitting mapping method, executed on an accelerator, includes the following steps:

[0083] Step S1: Split the mantissa field of each matrix of the input single-precision floating-point data into multiple low-bit-width integer matrices;

[0084] Step S2: Perform a fixed linear combination operation on the multiple sets of low-width integer matrices to generate multiple sets of point value matrices;

[0085] Step S3: Perform low-bit-width integer matrix multiplication on the multiple sets of point value matrices to obtain multiple sets of point value product matrices;

[0086] Step S4: Perform interpolation and weighted merging on the multiple sets of point-value product matrices to obtain the mantissa field product result of single-precision floating-point matrix multiplication;

[0087] Step S5: Combine the mantissa product result with the exponent field of the single-precision floating-point number to output the final single-precision floating-point matrix multiplication result.

[0088] In one embodiment, reference Figure 5 As shown, Figure 5 A detailed flowchart of step S4 is shown. The step of performing interpolation and weighted merging on the multiple sets of point-value product matrices to obtain the mantissa field product result further includes:

[0089] Step S41: Perform interpolation operations on the multiple sets of point value product matrices to obtain multiple sets of interpolation component matrices;

[0090] Step S42: Weight and merge the multiple sets of interpolation component matrices to obtain the mantissa product result of single-precision floating-point matrix multiplication.

[0091] In one embodiment, in step S1, the mantissa field of each matrix data in the single-precision floating-point data is split into three groups of 8-bit integer matrices according to a preset splitting radix, resulting in three groups of low-width integer matrices; wherein the splitting radix is ​​2... 0 2 -8 2 -16 .

[0092] In one embodiment of the present invention, in step S2, a linear combination is performed on three sets of low-width integer matrices using a preset evaluation point set {0,1,-1,2,∞} to generate five sets of point value matrices corresponding to the evaluation point set. The linear combination is achieved only through addition, subtraction and shift operations. The multiple sets of point value matrices are five sets.

[0093] In one embodiment of the present invention, in step S3, the five sets of point value matrices are paired up and low bit-width integer matrix multiplication is performed on each pair to obtain the corresponding five sets of point value product matrices.

[0094] It should be noted that the relevant technical details mentioned in the above embodiments are still valid in this method embodiment, and will not be repeated here to reduce repetition.

[0095] In summary, existing accelerators suffer from energy efficiency bottlenecks due to the mismatch between FP32 and INT8 computing resources, as well as efficiency limitations caused by excessive high-precision mapping overhead. This invention improves the accelerator by employing a matrix-level splitting mapping based on Toom-Cook-3 (TC-3). This maps FP32 matrix multiplication to fewer low-bit-width integer matrix multiplications through a fixed evaluation / interpolation linear transformation. This improves the reuse efficiency of existing low-bit-width arrays while maintaining FP32 semantics and accuracy requirements, allowing FP32 matrix multiplication to more fully utilize existing low-bit-width array resources, thereby improving overall throughput and energy efficiency. Simultaneously, it reduces mapping computation and system overhead, decreases the number of sub-operations and array calls, and reduces scheduling switching and intermediate data transfer overhead.

[0096] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A single-precision floating-point matrix multiplication calculation unit, characterized in that, include: The preprocessing module is used to read single-precision floating-point data and split the mantissa of each matrix into multiple low-bit-width integer matrices. A linear evaluation arithmetic logic module, connected to the preprocessing module, is used to perform linear combination operations on the multiple sets of low-bit-width integer matrices to generate multiple sets of point-value matrices. A low-bit-width integer computation array is connected to the linear evaluation arithmetic logic module and is used to perform low-bit-width integer matrix multiplication operations on the multiple sets of point-value matrices to obtain multiple sets of point-value product matrices. The interpolation and merging module is connected to the low-bit-width integer calculation array and is used to perform interpolation operations and weighted merging on the multiple sets of point-value product matrices to generate the mantissa field product result. The format conversion module, connected to the interpolation merging module, is used to combine the mantissa field product result with the exponent field part of each matrix to generate a single-precision floating-point matrix multiplication result.

2. The single-precision floating-point matrix multiplication calculation unit according to claim 1, characterized in that, The preprocessing module includes a mantissa splitting module and a radix configuration register, wherein the radix configuration register pre-stores 2 0 2 -8 2 -16 The mantissa splitting module splits the mantissa field of each matrix data in the single-precision floating-point data into three groups of 8-bit integer matrices based on the three splitting radixes, resulting in three groups of low-bit-width integer matrices.

3. The single-precision floating-point matrix multiplication calculation unit according to claim 1, characterized in that, The linear evaluation arithmetic logic module includes multiple sets of parallel adders / subtractors and shifters. It performs linear combinations on three sets of low-bit-width integer matrices using a preset evaluation point set {0,1,-1,2,∞} to generate five sets of point value matrices corresponding to the evaluation point set. The linear combination is implemented only through addition, subtraction and shift operations.

4. The single-precision floating-point matrix multiplication calculation unit according to claim 1, characterized in that, The low-bit-width integer computing array is the accelerator's native 8-bit integer matrix multiplication array; The five sets of point-value matrices are paired up and subjected to low-bit-width integer matrix multiplication to obtain the corresponding five sets of point-value product matrices.

5. The single-precision floating-point matrix multiplication calculation unit according to claim 1, characterized in that, The interpolation merging module includes a coefficient register, an interpolation module, and a weighted addition tree. The coefficient register pre-stores fixed coefficients for interpolation operations and predetermined base weights. The interpolation module performs interpolation operations on five sets of point value product matrices to generate five sets of interpolation component matrices. The weighted addition tree is shifted and summed according to the predetermined base weights to obtain the mantissa field product result.

6. The single-precision floating-point matrix multiplication calculation unit according to claim 1, characterized in that, It also includes: a global control module with built-in pipeline scheduling logic, which synchronizes the clock cycles of each unit via a control bus. While the preprocessing module splits the N+1th matrix, the low-bit-width integer calculation array performs the multiplication operation of the Nth matrix, and the interpolation merging module processes the product result of the N-1th matrix.

7. An AI accelerator, comprising at least a computing unit and a multi-level cache, characterized in that, The computing unit includes: The preprocessing module is used to read single-precision floating-point data and split the mantissa of each matrix into multiple low-bit-width integer matrices. A linear evaluation arithmetic logic module, connected to the preprocessing module, is used to perform linear combination operations on the multiple sets of low-bit-width integer matrices to generate multiple sets of point-value matrices. A low-bit-width integer computation array is connected to the linear evaluation arithmetic logic module and is used to perform low-bit-width integer matrix multiplication operations on the multiple sets of point-value matrices to obtain multiple sets of point-value product matrices. The interpolation and merging module is connected to the low-bit-width integer calculation array and is used to perform interpolation operations and weighted merging on the multiple sets of point-value product matrices to generate the mantissa field product result. The format conversion module, connected to the interpolation merging module, is used to combine the mantissa field product result with the exponent field part of each matrix to generate a single-precision floating-point matrix multiplication result and transmit it to the multi-level cache. The multi-level cache stores single-precision floating-point data and the results of single-precision floating-point matrix multiplication.

8. A single-precision floating-point matrix multiplication splitting mapping method, executed on an accelerator, characterized in that, include: The mantissa portion of each matrix of the input single-precision floating-point data is split into multiple low-bit-width integer matrices. Perform a fixed linear combination operation on the multiple sets of low-width integer matrices to generate multiple sets of point value matrices; Perform low-bit-width integer matrix multiplication on the multiple sets of point-value matrices to obtain multiple sets of point-value product matrices; Interpolation and weighted merging are performed on the multiple sets of point-value product matrices to obtain the mantissa field product result of single-precision floating-point matrix multiplication; The mantissa product result is combined with the exponent field of the single-precision floating-point number to output the final single-precision floating-point number. Precision floating-point matrix multiplication results.

9. The method according to claim 8, characterized in that, The step of performing interpolation and weighted merging on the multiple sets of point-value product matrices to obtain the mantissa field product result further includes: Interpolation operations are performed on the multiple sets of point value product matrices to obtain multiple sets of interpolation component matrices; The multiple interpolation component matrices are weighted and merged to obtain the mantissa product of single-precision floating-point matrix multiplication.

10. The method according to claim 8, characterized in that, Based on a preset splitting radix, the mantissa field of each matrix data in the single-precision floating-point data is split into three groups of 8-bit integer matrices, resulting in three groups of low-bit-width integer matrices; wherein the splitting radix is ​​2... 0 2 -8 2 -16 .

11. The method according to claim 8, characterized in that, By performing a linear combination on three sets of low-bit-width integer matrices using a preset evaluation point set {0,1,-1,2,∞}, five sets of point value matrices corresponding to the evaluation point set are generated. The linear combination is achieved only through addition, subtraction, and shift operations. The multiple sets of point value matrices consist of five sets.

12. The method according to claim 11, characterized in that, Pair the five sets of point value matrices together and perform low-bit-width integer matrix multiplication on each pair to obtain the corresponding five sets of point value product matrices.

Images