Optimization method for implementing high-performance single-precision matrix multiplication by using ascend-based half-precision computing units, and related device

Abstract

Provided in the embodiments of the present application are an optimization method for implementing high-performance single-precision matrix multiplication by using Ascend-based half-precision computing units, and a related device. The method comprises: first, acquiring a first half-precision matrix of a first single-precision matrix, a second half-precision matrix of a second single-precision matrix, a second error matrix between the second single-precision matrix and the second half-precision matrix, and a plurality of second block error matrices in the second error matrix, and storing the second block error matrices in an L1 buffer of parallel computing hardware; and acquiring first block matrices in the first half-precision matrix one by one, storing the first block matrices in the L1 buffer, further performing a matrix operation to obtain a first result matrix, acquiring a second result matrix and a third result matrix, summing the first result matrix, the second result matrix and the third result matrix to obtain a single-precision target matrix, and using the single-precision target matrix as the result of performing a matrix multiplication operation on the first single-precision matrix and the second single-precision matrix. Thus, the accuracy and computational speed of single-precision matrix multiplication are effectively improved.

Description

An optimization method for high-performance single-precision matrix multiplication implemented by a semi-precision computing unit based on Ascend and related equipment TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, and in particular to an optimization method for high-performance single-precision matrix multiplication implemented by a semi-precision computing unit based on Ascend and related equipment. BACKGROUND

[0002] With the continuous development of the fields of neural networks, scientific computing and artificial intelligence, these disciplines have gradually been applied to life and work in recent years. As the input feature maps used in these fields become larger and the number of layers and coupling relationships of the models used become more complex, the performance of the calculations has become one of the indicators that these fields focus on. Matrix multiplication operations are used a lot in these disciplines, so the performance improvement of matrix multiplication operations is crucial.

[0003] In related technologies, for single-precision matrix multiplication operations, a computing chip that can support single-precision calculations is usually used for further calculations, but the computing chip that supports single-precision calculations has a high cost. In addition, in the use of a computing chip that only supports semi-precision calculations, the single-precision matrix to be multiplied is converted into a semi-precision matrix, and then semi-precision multiplication is performed, but the accuracy of the results obtained by this scheme is low.

[0004] SUMMARY

[0005] The optimization method for high-performance single-precision matrix multiplication implemented by a semi-precision computing unit based on Ascend and related equipment provided by the embodiments of the present application can improve the accuracy of single-precision matrix multiplication operations in the use of a computing chip that supports semi-precision calculations.

[0006] To achieve the above object, a first aspect of the embodiments of the present application proposes an optimization method for high-performance single-precision matrix multiplication implemented by a semi-precision computing unit based on Ascend, which comprises:

[0007] obtaining a first semi-precision matrix of a first single-precision matrix and a second semi-precision matrix of a second single-precision matrix;

[0008] obtaining a second error matrix between the second single-precision matrix and the second semi-precision matrix, obtaining a plurality of second block error matrices in the second error matrix, and storing them in an L1 buffer of the parallel computing hardware, the second error matrix being a semi-precision matrix;

[0009] acquire the first block matrix in the first half-precision matrix one by one, store the first block matrix in the L1 buffer, transmit the first block matrix in the L1 buffer and the second block error matrix to a matrix calculation module of the parallel computing hardware for matrix multiplication, obtain a first block result matrix, and obtain a first result matrix of the first half-precision matrix and the second error matrix based on the first block result matrix;

[0010] acquire a first error matrix between the first single-precision matrix and the first half-precision matrix, the first error matrix being a half-precision matrix;

[0011] acquire a second result matrix obtained by matrix multiplication of the first error matrix and the second half-precision matrix, acquire a third result matrix obtained by matrix multiplication of the first half-precision matrix and the second half-precision matrix, accumulate the first result matrix, the second result matrix, and the third result matrix to obtain a single-precision target matrix, and take the single-precision target matrix as a result of matrix multiplication of the first single-precision matrix and the second single-precision matrix.

[0012] In some embodiments, the acquiring a plurality of second block error matrices in the second error matrix and storing the second block error matrices in the L1 buffer of the parallel computing hardware comprises:

[0013] determine a first storage quantity based on a first block size of the first block matrix and a second block size of the second block error matrix;

[0014] store the second block error matrix corresponding to the first storage quantity in the L1 buffer.

[0015] In some embodiments, the determining a first storage quantity based on a first block size of the first block matrix and a second block size of the second block error matrix comprises:

[0016] obtain an error matrix storage capacity based on a difference between a cache capacity of the L1 buffer and twice the first block size;

[0017] obtain the first storage quantity based on a ratio of the error matrix storage capacity and the second block size.

[0018] In some embodiments, the L1 buffer is provided with a first storage location and a second storage location for storing the first block matrix, and the acquiring the first block matrix in the first half-precision matrix one by one, storing the first block matrix in the L1 buffer, transmitting the first block matrix in the L1 buffer and the second block error matrix to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first block result matrix comprises:

[0019] According to the arrangement order of the plurality of first sub-block matrices in the first half-precision matrix, a second first sub-block matrix is obtained and stored in the first storage location;

[0020] The first first sub-block matrix in the second storage location and the second sub-block error matrix corresponding to the first first sub-block matrix in the L1 buffer are transmitted to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first sub-block result matrix.

[0021] In some embodiments, the transmitting the first sub-block matrix and the second sub-block error matrix in the L1 buffer to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first sub-block result matrix comprises:

[0022] In the L1 buffer, the plurality of second sub-block error matrices are multiplied by an amplification factor to obtain amplified second sub-block error matrices;

[0023] The amplified second sub-block error matrix corresponding to the first sub-block matrix in the L1 buffer is transmitted to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain the first sub-block result matrix.

[0024] In some embodiments, before the obtaining a first half-precision matrix of a first single-precision matrix and a second half-precision matrix of a second single-precision matrix, the method further comprises:

[0025] Pipelining is locked for a sub-block error module, the sub-block error module comprising the generation and storage of the second sub-block error matrix to the L1 buffer;

[0026] After the obtaining a plurality of second sub-block error matrices in the second error matrix and storing them in the L1 buffer of the parallel computing hardware, the method further comprises:

[0027] The sub-block error module is unlocked from pipelining.

[0028] In some embodiments, before the obtaining a first sub-block matrix in the first half-precision matrix and storing it in the L1 buffer, the method further comprises:

[0029] Pipelining is locked for a sub-block calculation module, the sub-block calculation module comprising the storage of the first sub-block matrix to the L1 buffer and the matrix calculation process of the first sub-block matrix and the second sub-block error matrix;

[0030] After the obtaining the first half-precision matrix and the first result matrix of the second error matrix based on the first sub-block result matrix, the method further comprises:

[0031] The block calculation module is pipelined.

[0032] In some embodiments, the first result matrix of the first half-precision matrix and the second error matrix based on the first block result matrix includes:

[0033] The first half-precision matrix and the first result matrix of the second error matrix are obtained by accumulating the first block result matrix corresponding to each first block matrix.

[0034] In some embodiments, the second result matrix obtained by matrix multiplication of the first error matrix and the second half-precision matrix includes:

[0035] According to the first block size, a third block size is obtained, and according to the second block size, a fourth block size is obtained;

[0036] According to the third block size, the second half-precision matrix is split to obtain a plurality of second block matrices;

[0037] According to the fourth block size, the first error matrix is split to obtain a plurality of first block error matrices;

[0038] Each of the second block matrix and the first block error matrix is obtained through the L1 buffer and the matrix calculation module to obtain a second block result matrix, and the second result matrix is obtained based on all the second block result matrix.

[0039] To achieve the above-mentioned purpose, a second aspect of the embodiment of the present application proposes a data processing device for implementing high-performance single-precision matrix multiplication based on Ascend half-precision calculation unit, the device comprises:

[0040] The matrix acquisition module is configured to acquire a first half-precision matrix of a first single-precision matrix and a second half-precision matrix of a second single-precision matrix;

[0041] The first error matrix module is configured to acquire a second error matrix between the second single-precision matrix and the second half-precision matrix, acquire a plurality of second block error matrices in the second error matrix, and store them in the L1 buffer of the parallel calculation hardware, the second error matrix being a half-precision matrix;

[0042] The matrix calculation module is configured to acquire the first sub-block matrix in the first half-precision matrix one by one, store the first sub-block matrix in the L1 buffer, and transmit the first sub-block matrix in the L1 buffer and the second sub-block error matrix to the matrix calculation module of the parallel computing hardware to perform matrix multiplication, so as to obtain a first sub-block result matrix, and obtain a first result matrix of the first half-precision matrix and the second error matrix based on the first sub-block result matrix.

[0043] The second error matrix module is configured to acquire a first error matrix between the first single-precision matrix and the first half-precision matrix, and the first error matrix is a half-precision matrix.

[0044] The result calculation module is configured to acquire a second result matrix obtained by performing matrix multiplication on the first error matrix and the second half-precision matrix, acquire a third result matrix obtained by performing matrix multiplication on the first half-precision matrix and the second half-precision matrix, accumulate the first result matrix, the second result matrix and the third result matrix to obtain a single-precision target matrix, and take the single-precision target matrix as a result of matrix multiplication of the first single-precision matrix and the second single-precision matrix.

[0045] To achieve the above object, a third aspect of the embodiments of the present application provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the optimization method for realizing high-performance single-precision matrix multiplication of the Ascend-based half-precision calculation unit when executing the computer program.

[0046] To achieve the above object, a fourth aspect of the embodiments of the present application provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the optimization method for realizing high-performance single-precision matrix multiplication of the Ascend-based half-precision calculation unit as described in the first aspect.

[0047] The optimization method for high-performance single-precision matrix multiplication implemented by the half-precision calculation unit based on Ascend and related equipment provided by the embodiments of the present application, the method comprises the following steps: first, a first half-precision matrix of a first single-precision matrix and a second half-precision matrix of a second single-precision matrix are obtained; then, a second error matrix between the second single-precision matrix and the second half-precision matrix is obtained, a plurality of second block error matrices in the second error matrix are obtained, and the second error matrix is stored in an L1 buffer of parallel computing hardware, and the second error matrix is a half-precision matrix; then, a first block matrix in the first half-precision matrix is obtained one by one and stored in the L1 buffer, the first block matrix and the second block error matrix in the L1 buffer are transmitted to a matrix calculation module of the parallel computing hardware for matrix multiplication, a first block result matrix is obtained, a first result matrix of the first half-precision matrix and the second error matrix is obtained based on the first block result matrix; and a first error matrix between the first single-precision matrix and the first half-precision matrix is obtained, and the first error matrix is a half-precision matrix; finally, a second result matrix obtained by multiplying the first error matrix and the second half-precision matrix is obtained, a third result matrix obtained by multiplying the first half-precision matrix and the second half-precision matrix is obtained, a single-precision target matrix is obtained by accumulating the first result matrix, the second result matrix and the third result matrix, and the single-precision target matrix is taken as a result of the matrix multiplication of the first single-precision matrix and the second single-precision matrix. The embodiments of the present application utilize the error matrix generated after the single-precision matrix in the parallel computing hardware is converted to the half-precision matrix in advance, and then utilizes the product result of the error matrix and the half-precision matrix to compensate the precision in the multiplication process of the single-precision matrix, thereby effectively improving the precision of the single-precision matrix multiplication in the parallel computing hardware supporting half-precision calculation; in addition, the L1 buffer in the parallel computing hardware is taken as an intermediate buffer, the high capacity and low real-time transmission rate of the L1 buffer are combined, a large number of block matrices are transmitted to the L1 buffer in advance, so that the real-time data exchange transmission of the matrix data in the L1 buffer is effectively reduced in the actual calculation process, and the calculation rate of the matrix operation in the parallel computing hardware is effectively improved.

[0048] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent from the description, or can be learned by practice of the present application. The objects and other advantages of the present application will be realized and achieved by means of the structures particularly pointed out in the description and claims. BRIEF DESCRIPTION OF DRAWINGS

[0049] FIG. 1 is a structural schematic diagram of an AI chip according to an embodiment of the present application.

[0050] FIG. 2 is a flowchart of an optimization method for implementing high-performance single-precision matrix multiplication by a half-precision computing unit based on Ascend according to another embodiment of the present application.

[0051] FIG. 3 is a flowchart of single-precision matrix multiplication according to another embodiment of the present application.

[0052] FIG. 4 is a flowchart of a double-buffer optimization algorithm according to another embodiment of the present application.

[0053] FIG. 5 is a flowchart of a block operation of single-precision matrix multiplication according to another embodiment of the present application.

[0054] FIG. 6 is a flowchart of a multiplication operation of a half-precision matrix and an error matrix according to another embodiment of the present application.

[0055] FIG. 7 is a schematic diagram of a pipeline configuration according to another embodiment of the present application.

[0056] FIG. 8 is a flowchart of another multiplication operation of a half-precision matrix and an error matrix according to another embodiment of the present application.

[0057] FIG. 9 is a flowchart of double-buffer memory switching according to another embodiment of the present application.

[0058] FIG. 10 is a schematic diagram of performance simulation parameters of a data processing method according to another embodiment of the present application.

[0059] FIG. 11 is a performance simulation diagram of a first data processing method according to another embodiment of the present application.

[0060] FIG. 12 is a performance simulation diagram of a second data processing method according to another embodiment of the present application.

[0061] FIG. 13 is a performance simulation diagram of a third data processing method according to another embodiment of the present application.

[0062] FIG. 14 is a structural diagram of a data processing apparatus for implementing high-performance single-precision matrix multiplication by a half-precision computing unit based on Ascend according to an embodiment of the present application.

[0063] FIG. 15 is a hardware structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solutions and advantages of the present application clearer, further detailed description will be made to the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0065] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical sequence is shown in the flowchart, in some cases, the steps shown or described can be performed in a manner different from the module division in the device or the sequence in the flowchart.

[0066] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification herein is for describing the embodiments of the application only and is not intended to be limiting of the application.

[0067] First, several terms involved in the present application are analyzed:

[0068] Single-precision matrix multiplication refers to the multiplication of two matrices of single-precision floating-point type. Single-precision floating-point type usually uses 32-bit binary numbers to represent a floating-point number, of which 1 bit is used for the sign bit, 8 bits for the exponent, and 23 bits for the mantissa. Assuming that the size of matrix A is m x n and the size of matrix B is n x p, then the size of their product C is m x p, where each element C[i][j] of matrix C is the sum of the product of the i-th row of matrix A and the j-th column of matrix B.

[0069] Similar to single-precision matrix multiplication, half-precision matrix multiplication refers to the multiplication of two matrices of half-precision floating-point type. Half-precision floating-point type usually uses 16-bit binary numbers to represent a floating-point number, of which 1 bit is used for the sign bit, 5 bits for the exponent, and 10 bits for the mantissa. Because the precision of half-precision floating-point type is lower, in actual application, half-precision matrix multiplication is usually used in scenarios that require less precision, such as calculations in neural networks.

[0070] IEEE-754 is a binary floating-point representation method, which is an international standard developed by the Institute of Electrical and Electronics Engineers (IEEE). The standard specifies the binary representation method of floating-point numbers in computers, including the representation of sign bits, exponent bits, and mantissa bits.

[0071] Parallel computing hardware refers to a hardware device that can perform multiple computing tasks simultaneously. These hardware devices usually have high parallel computing capabilities and efficient computing resource management capabilities, and can achieve efficient computing and data processing.

[0072] With the continuous development of neural network, scientific computing and artificial intelligence fields, these disciplines have gradually been applied to life and work in recent years. As the input feature map used in these fields becomes larger and larger, the number of layers and coupling relationship of the model used becomes more and more complex, the computing performance has become one of the indicators that these fields focus on. Among these disciplines, matrix multiplication operation is used a lot, therefore, the performance improvement of matrix multiplication operation is crucial.

[0073] In the related art, for single-precision matrix multiplication operation, a calculation chip supporting single-precision calculation is usually used for further calculation, but the calculation chip supporting single-precision calculation has a high cost. In addition, in the use of a calculation chip supporting only half-precision calculation, the single-precision matrix to be multiplied is converted into a half-precision matrix, and then the half-precision is multiplied, but the accuracy of the result obtained by this scheme is low.

[0074] In order to improve the accuracy of single-precision matrix multiplication operation in the use of calculation chip supporting half-precision calculation, the embodiments of the present application utilize the error matrix generated by the single-precision matrix converted into half-precision matrix in advance in parallel computing hardware, and then utilizes the product result of the error matrix and the half-precision matrix to compensate the accuracy in the process of single-precision matrix multiplication operation, thereby effectively improving the accuracy of single-precision matrix multiplication in the use of parallel computing hardware supporting half-precision calculation. In addition, the L1 buffer in parallel computing hardware is used as an intermediate buffer, and the high capacity and low real-time transmission rate of the L1 buffer are combined to pre-transfer a large number of block matrices to the L1 buffer, so that in the actual calculation process, the real-time data exchange transmission of matrix data in the L1 buffer is effectively reduced, and the calculation rate of matrix operation in parallel computing hardware is effectively improved.

[0075] To better describe the optimization method for implementing high-performance single-precision matrix multiplication by the half-precision computing unit based on Ascend provided in the present application, the parallel computing hardware applied to the optimization method for implementing high-performance single-precision matrix multiplication by the half-precision computing unit based on Ascend is first described below. The parallel computing hardware includes multiple AI chips that are connected through communication serial ports. Referring to FIG. 1, it is a structural schematic diagram of an AI chip provided in an embodiment of the present application. As shown in FIG. 1, the AI chip includes multiple data storage spaces and computing units. The data storage spaces include a global memory (Blobal Memory) that usually has a data storage space of 1 GB, a unified buffer (Unified Buffer, UB) that usually has a data storage space of 256 KB, an L1 buffer that usually has a data storage space of 1024 KB, an L0A storage unit and an L0B storage unit that usually have a data storage space of 64 KB, an L0C storage unit that usually has a data storage space of 256 KB, and a storage conversion unit. The computing units include a matrix computing unit (CUBE Unit), an accumulator (Accelerator), a vector computing unit (Vector Unit), and a scalar computing unit (Scalr Unit). In a typical computing process, two input matrices to be multiplied are input from the global memory and sequentially pass through the unified buffer, the storage conversion unit, the L1 buffer, the storage conversion unit, and then are input into the L0A storage unit and the L0B storage unit, respectively. Then, the matrix computing unit is used to perform multiplication operation on the two input matrices, and the matrix multiplication result is input into the L0C storage unit.

[0076] The AI chip described above only supports half-precision matrix multiplication. For example, the 910A model of Ascend AI processor is a typical chip that only supports half-precision matrix multiplication. When this type of chip uses FP16 to simulate FP32 matrix multiplication, the operation process is the same as that of many chips specially accelerated for matrix multiplication, and the process is as follows:

[0077] 1. Data transfer: transfer the FP32 matrices A and B from the main memory GM to the UB;

[0078] 2. Data calculation: convert the FP32 format matrices A and B into the FP16 format on the VEC module;

[0079] 3. Data transfer: transfer the FP16 format matrices A and B to L0A and L0B;

[0080] 4. Data calculation: perform FP16 format matrix multiplication and addition operation on the CUBE module;

[0081] 5. Data transfer: Transfer the calculation results to the main memory (GM).

[0082] When the data volume of matrix operations is particularly large, the size of the data blocks becomes a limiting factor for performance improvement due to the limitations of the processor's on-chip cache. Inefficient data transfer also becomes a limiting factor for performance improvement due to the unreasonable allocation of time between data transfer and computation. Furthermore, because the matrix multiplication operation process on the Ascend AI processor is basically fixed, pipeline optimization for specific algorithms requires separate handling, making the conventional optimization methods provided by the processor inapplicable.

[0083] Based on the aforementioned parallel computing hardware, the following will further describe the optimization method and related equipment for implementing high-performance single-precision matrix multiplication using the Ascend-based half-precision computing unit provided in this application embodiment. The optimization method for implementing high-performance single-precision matrix multiplication using the Ascend-based half-precision computing unit provided in this application embodiment can be applied to this parallel computing hardware, and also to the AI ​​chip within this parallel computing hardware.

[0084] The following describes in detail the optimization method for implementing high-performance single-precision matrix multiplication using the Ascend-based half-precision computing unit in the embodiments of this application. Referring to Figure 2, an optional flowchart of the optimization method for implementing high-performance single-precision matrix multiplication using the Ascend-based half-precision computing unit provided in the embodiments of this application is shown. The method in Figure 2 may include, but is not limited to, steps 100 to 500. It is also understood that this embodiment does not specifically limit the order of steps 100 to 500 in Figure 2, and the order of steps can be adjusted or certain steps can be reduced or added according to actual needs.

[0085] Step 100: Obtain the first half-precision matrix of the first single-precision matrix and the second half-precision matrix of the second single-precision matrix.

[0086] Step 100 is described in detail below.

[0087] In some embodiments, when the parallel computing hardware responds to the first single-precision matrix A single Second single-precision matrix B single During multiplication operations, the AI ​​chip in the parallel computing hardware cannot directly handle matrix multiplication of single-precision matrices. Therefore, the parallel computing hardware first multiplies the first single-precision matrix A... single Second single-precision matrix B single The input is taken from the global buffer to the unified buffer UB, and then the vector computation unit Vec processes the first single-precision matrix A. single Second single-precision matrix B single Perform a half-precision matrix transformation to obtain the first single-precision matrix A. single The first half-precision matrix Ahalf and the second half-precision matrix B single of the second half-precision matrix B half and then the first half-precision matrix A half and the second half-precision matrix B half are used to perform matrix multiplication.

[0088] In order to further improve the accuracy of the matrix multiplication of the two single-precision matrices on the parallel computing hardware, the first single-precision matrix A single is converted into the first half-precision matrix A half and the first error matrix ΔA half generated in the process, and the second single-precision matrix B single is converted into the second half-precision matrix B half and the second error matrix ΔB half generated in the process are used as compensation terms to improve the accuracy of the matrix multiplication.

[0089] Referring to FIG. 3, it is a flowchart of the single-precision matrix multiplication provided by the embodiment of the present application. As shown in FIG. 3, the multiplication operation of the first single-precision matrix A single and the second single-precision matrix B single is divided into three parts. Among them, it includes the multiplication operation between the first half-precision matrix A half and the second half-precision matrix B half , the multiplication operation between the first half-precision matrix A half and the second error matrix ΔB half and the multiplication operation between the second half-precision matrix B half and the first error matrix ΔA half , and the calculation result of the multiplication operation between the first single-precision matrix A single and the second single-precision matrix B single is obtained after accumulation.

[0090] It can be understood that when the matrix dimensions of the first single-precision matrix A single and the second single-precision matrix B single are too large, the corresponding first half-precision matrix A half , second half-precision matrix B half , first error matrix ΔA half and second error matrix ΔB half cannot be stored in the L0A storage unit and L0B storage unit in the AI chip due to the limited storage space, and therefore it is necessary to divide the matrix into multiple block matrices (for example, the first half-precision matrix A half is divided into multiple first block matrices ), and sequentially input the first single-precision matrix A single and the second single-precision matrix B single into the matrix calculation unit to obtain the multiplication result C single of the first single-precision matrix A single and the second single-precision matrix B single .

[0091] Step 200: obtaining a second error matrix between the second single-precision matrix and the second half-precision matrix, obtaining a plurality of second block error matrices in the second error matrix, and storing the plurality of second block error matrices into an L1 buffer of the parallel computing hardware.

[0092] The step 200 is described in detail as follows.

[0093] In some embodiments, through the flow analysis of the matrix multiplication as shown in FIG. 3, it can be seen that the data transfer and the data calculation are interleaved. If the size of the input matrix (such as the first single-precision matrix A single and the second single-precision matrix B single ) is particularly large, it needs to be divided into blocks. If the performance is to be improved, the data needs to be allocated on the multi-core in advance. The data blocking and the use of multi-core calculation both increase the complexity of the data transfer and the data calculation, resulting in the coupling of the data transfer and the data calculation. At this time, if the data transfer is relatively slow compared with the calculation (that is, the memory access wall problem occurs), the processor of the AI chip often needs to wait for reading data from the memory or writing data into the memory, resulting in an increase in the idle waiting time of the processor. In addition, since the AI chip (such as the Ascend processor) provides two levels of cache: the L0 buffer and the L1 buffer, the data buffering rate of the L0 buffer to the global buffer is much faster than the rate between the L1 buffer and the global cache, resulting in a bottleneck of the data transfer in the L1 buffer.

[0094] It can be understood that the memory access wall problem is for the matrix multiplication operation, which means that in the computer system, the CPU processing speed is much faster than the memory access speed in improving the performance, resulting in an increase in the waiting time of the processor, and the improvement of the matrix multiplication performance is mainly restricted by the memory access speed.

[0095] In combination with the fact that the cache capacity of the L1 buffer on the AI chip is 1 MB, which is much larger than the cache capacity of the uniform buffer UB of 256 KB, and the matrix blocking and the vector calculation unit VEC need to frequently use the uniform buffer UB, the L1 buffer is used as the intermediate cache in the matrix operation process in the embodiment. This will be described in detail further.

[0096] Since the operation flow of the three multiplication parts as shown in FIG. 3 in the AI chip is similar, in the present embodiment, the multiplication operation between the first half-precision matrix A half and the second error matrix ΔB half will be described in detail first, and the remaining two parts can be similarly obtained.

[0097] In some embodiments, the second single-precision matrix B single is converted into multiplication of the second half-precision matrix B half , and then the second single-precision matrix B single is subtracted from the second half-precision matrix B half to obtain the second single-precision error matrix ΔB single = B single -to-single(B half ), and the second single-precision error matrix is converted into half-precision to obtain the second error matrix ΔB half , in the case of assuming that the second single-precision matrix B single is too large, the second error matrix ΔB half includes a plurality of second block error matrices where K is the number of matrixes of the second block error matrices stored in the L1 buffer of the parallel computing hardware at one time.

[0098] Then, before the multiplication operation between the first half-precision matrix A half and the second error matrix ΔB half , the plurality of second block error matrices are pre-stored in the L1 buffer of the parallel computing hardware, so that in the actual multiplication operation between the first half-precision matrix A half and the second error matrix ΔB half , the data storage and transfer flow of the L1 buffer is reduced to overcome the memory wall problem mentioned above, and the burden of the unified buffer UB is reduced, thereby effectively improving the calculation rate of matrix operation data processing in the actual process.

[0099] Wherein, obtaining the plurality of second block error matrices in the second error matrix and storing them in the L1 buffer of the parallel computing hardware includes the following steps 210 to 220.

[0100] Step 210: Determine the first storage number based on the first block size of the first block matrix and the second block size of the second block error matrix.

[0101] The step 210 will be described in detail below.

[0102] In some embodiments, the L1 buffer is used as an intermediate cache in the matrix operation process, and is used to pre-store a plurality of second sub-block error matrices before the multiplication operation between the first half-precision matrix A half and the second error matrix ΔB half . To further improve the utilization of the storage space of the L1 buffer, the L1 buffer needs to be appropriately divided into a plurality of storage spaces in advance, including the number of spaces for storing the second sub-block error matrix and the number of spaces for storing the first sub-block matrix.

[0103] Therefore, in the present embodiment, based on the first sub-block size b m ×b k of the first sub-block matrix and the second sub-block size b k ×b n of the second sub-block error matrix , the first storage number N L1 for storing the number of second sub-block error matrices in the L1 buffer is determined.

[0104] Wherein, the first storage number is determined based on the first sub-block size of the first sub-block matrix and the second sub-block size of the second sub-block error matrix, including the following steps 211 to step 212.

[0105] Step 211: based on the difference between the cache capacity of the L1 buffer and twice the number of first sub-block sizes, the error matrix storage capacity is obtained.

[0106] Step 212: based on the ratio of the error matrix storage capacity and the second sub-block size, the first storage number is obtained.

[0107] The steps 211 to step 212 are described in detail below.

[0108] In some embodiments, before the multiplication operation between the first half-precision matrix A half and the second error matrix ΔB half , first, based on the difference between the cache capacity of the L1 buffer (i.e. 1024KB) and twice the number of first sub-block sizes 2b m b k , the error matrix storage capacity 1024-2b m b k is obtained. Next, based on the ratio of the error matrix storage capacity and the second sub-block size b k b n , the first storage number N L1 = (1024-2b m b k ) / (bk b n )。

[0109] from the first storage number N L1 It can be seen from the formula that N L1 +2 storage spaces are divided in the L1 buffer, wherein N L1 storage spaces are used to pre-store the second block error matrix The other 2 storage spaces are used to store the first block matrix half during the multiplication operation between the first half-precision matrix A half and the second error matrix ΔB L1 .

[0110] Step 220: Store the second block error matrix corresponding to the first storage number into the L1 buffer.

[0111] The steps 210 to 220 are described in detail as follows.

[0112] In some embodiments, after the appropriate storage space is divided in the L1 buffer, the second block error matrix corresponding to the first storage number (i.e., N L1 ) is pre-stored into the L1 buffer for intermediate caching.

[0113] In addition, for a column in the second error matrix ΔB half , it is divided into K / N L1 groups, and the number of the second block error matrix in each group is N L1 . All the second block error matrices in each group are uniformly cleaned up and transported into the L1 buffer at one time to complete the optimization task of multiplexing the L1 buffer as an intermediate cache.

[0114] It can be understood that the second error matrix ΔB half is traversed first because the calculation complexity of obtaining the second error matrix ΔB half is higher than that of obtaining the first half-precision matrix A half in the multiplication operation between the first half-precision matrix A half and the second error matrix ΔB half . Therefore, in this embodiment, in the multiplication operation between the first half-precision matrix A half and the second error matrix ΔB half , only the vector calculation unit VEC is involved in the calculation of the second error matrix ΔB half , which is separated from the calculation of the matrix calculation unit CUBE, so that the calculation of the first half-precision matrix Ahalf and the second error matrix ΔB half The calculation speed of the matrix multiplication operation.

[0115] In addition, by using the L1 buffer as the intermediate cache for multiplexing in the multiplication operation of the block matrix, the pressure on the use of the unified buffer UB can be effectively reduced, and by using the larger storage space of the L1 buffer, larger matrix blocks can be obtained when the matrix is divided, and larger blocks can reduce the number of data transfers and the number of calls to the matrix calculation unit CUBE, thereby facilitating the speed of the matrix multiplication operation. And by using the L1 buffer as the intermediate cache for multiplexing in the multiplication operation of the block matrix, more data can be stored at one time, that is, most of the storage space is used to store the second error matrix in advance, so that the number of data transfer and data calculation cross execution is reduced, which is more conducive to the performance of matrix multiplication, and further effectively improves the calculation speed of the matrix multiplication operation of the first half-precision matrix and the second error matrix, to improve the calculation speed of the matrix operation in parallel computing hardware.

[0116] Step 300: Obtain the first block matrix in the first half-precision matrix one by one, store it in the L1 buffer, and transmit the first block matrix and the second block error matrix in the L1 buffer to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain the first block result matrix, and obtain the first result matrix of the first half-precision matrix and the second error matrix based on the first block result matrix.

[0117] The step 300 is described in detail below.

[0118] In some embodiments, after the plurality of second block error matrices are stored in the L1 buffer in advance, the actual matrix operation of the second block error matrix and the first block matrix needs to be performed.

[0119] Next, the first block matrix half in the first half-precision matrix A is obtained one by one and stored in the L1 buffer, and the first block matrix and the second block error matrix in the L1 buffer are transmitted to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain the first block result matrix.

[0120] When a single block matrix is buffered in the L1 buffer, the data transfer and data calculation of the block matrix are strictly in series. Taking the calculation of the first half-precision matrix and the second error matrix as an example, the first block matrix Second block error matrix Each data item is moved to a unified buffer (UB), and then cleaned by the vector computation unit (VEC) to obtain the FP16 format. and Then, the matrix computation unit CUBE is called to perform matrix multiplication and addition operations to obtain the block calculation results, which are then written back to the global cache. Each calculation step waits for the data transfer from the previous step to complete, which severely impacts computational efficiency.

[0121] Therefore, in this embodiment of the application, based on the aforementioned first storage quantity N L1 As can be seen from the formula, the L1 buffer is used for the first block matrix. Two storage spaces were allocated, including a first storage location and a second storage location, for the first block matrix. The cache is configured with a dual cache space to separate data handling and data computation, thereby enabling parallel processing of data handling and data computation to accelerate matrix multiplication.

[0122] The process includes steps 310 to 320, which involve obtaining the first block matrix from the first half-precision matrix one by one, storing it in the L1 buffer, and then transmitting the first block matrix and the second block error matrix in the L1 buffer to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain the first block result matrix.

[0123] Step 310: Based on the arrangement order of the multiple first block matrices in the first half-precision matrix, obtain the second first block matrix and store it in the first storage location.

[0124] Step 320: Transmit the first block matrix in the second storage location and the second block error matrix corresponding to the first block matrix in the L1 buffer to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain the first block result matrix.

[0125] Steps 310 to 320 are described in detail below.

[0126] In some embodiments, firstly based on the first half-precision matrix A half Multiple first block matrices The order of arrangement, one by one, the first block matrix Stored to the first or second storage location in the L1 buffer, and then in the next first block matrix. When storing data to a specific location in the L1 buffer, the first block matrix previously stored at another location is also stored. It can be executed with the corresponding second block error matrix. The process of transmitting the data to the matrix computation unit CUBE for matrix multiplication. That is, multiplying the second first-block matrix. When storing data to the first (or second) storage location in the L1 buffer, in parallel-running hardware, the first block matrix of the second (or first) storage location in the L1 buffer will be executed synchronously. and the L1 buffer with the first block matrix The corresponding second block error matrix The matrix computation module CUBE, which transmits the data to the parallel computing hardware, performs matrix multiplication to obtain the first block matrix. The corresponding second block error matrix The first block result matrix is ​​obtained by performing multiplication operations between them.

[0127] Referring to Figure 4, it is a schematic diagram of the algorithm flow for double-buffering optimization provided in an embodiment of this application. As shown in Figure 4, it lists the relevant implementation code for steps 310 to 320, wherein the block matrix of the next first single-precision matrix is... Move the block matrix from the global cache (GM) to the unified buffer (UB). Convert to the corresponding first block matrix And further divide the first block matrix The data is transferred from the unified buffer UB to the first storage location in the L1 buffer, then further transferred to the L0A storage unit, and the previous first block matrix is ​​also transferred. With the corresponding second block error matrix In the matrix computation module CUBE, matrix multiplication is performed to obtain the first block result matrix. That is, for N... L1 indivual N L1 indivual The blocks are divided into two groups, numbered sequentially starting from 0. Even-numbered sequences use the first reserved storage location in the L1 buffer (marked as L1). A0 Odd-numbered indices use the second storage location reserved in the L1 buffer (marked as L1). A1 Thus, when L1 A0 Provides the necessary information for the current matrix multiplication calculation. At that time, L1 A1 Can read the next one This enables data transfer and data computation to be performed simultaneously. These commands (i.e., the algorithm lines within the black box in Figure 4) can be implemented at the same time through the double buffer space setting in the L1 buffer, thereby effectively improving the computational efficiency of matrix multiplication in the parallel computing unit.

[0128] In a conventional FP16 simulation FP32 matrix multiplication algorithm, for a block calculation result, A half x ΔB half and ΔA half x B half are mixed in the calculation. As shown in a flowchart analysis of a single-precision matrix multiplication in FIG. 3, because the values of the first error matrix ΔA half and the second error matrix ΔB half are too small, the first block error matrix ΔA and the second block error matrix ΔB need to be multiplied by a scaling factor SF before multiplication, so as to reduce the error. However, this flowchart needs to perform many times of multiplication by the scaling factor SF, thereby reducing the calculation efficiency of matrix multiplication in parallel computing hardware. In this embodiment, in order to solve this problem, the L1 buffer is used to pre-divide a plurality of storage spaces to improve the calculation efficiency, and the specific process is as follows.

[0129] In the process, the first block matrix and the second block error matrix in the L1 buffer are transmitted to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain the first block result matrix, and the process further includes the following steps 330 to 340.

[0130] Step 330: In the L1 buffer, multiply the plurality of second block error matrices by the scaling factor to obtain the scaled second block error matrices.

[0131] Step 340: Transmit the scaled second block error matrices corresponding to the first block matrix in the L1 buffer to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain the first block result matrix.

[0132] The steps 330 to 340 are described in detail as follows.

[0133] In some embodiments, in the operation of A half x ΔB half , a plurality of storage spaces are pre-divided in the L1 buffer, and a plurality of second block error matrices ΔB are pre-cached, then the plurality of second block error matrices ΔB in the L1 buffer are multiplied by the scaling factor SF at the same time to change the plurality of second block error matrices ΔB in the L1 buffer into the corresponding scaled second block error matrices ΔB , thereby effectively reducing the operation of multiplying the scaling factor in the actual A half x ΔB half operation.

[0134] After obtaining the plurality of amplified second sub-block error matrices Then, the first sub-block matrix in the L1 buffer is transmitted to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain the corresponding first sub-block result matrix. The corresponding amplified second sub-block error matrix is multiplied by the amplification factor SF, and then the multiplication calculation process is performed.

[0135] Referring to FIG. 5, it is a flowchart of the single-precision matrix multiplication block operation provided by the embodiment of the application. As shown in FIG. 5, two second sub-block error matrices or two first sub-block error matrices are pre-stored in the L1 buffer. In the operation flow, the two second sub-block error matrices (or two first sub-block error matrices ) are multiplied by the amplification factor SF before the multiplication calculation process. As can be seen from the multiplication calculation process shown in FIG. 5, the amplification factor SF participates in 5 calculations (1 calculation for each reuse of the L1 buffer cache, 4 calculations for 4 reuses of the L1 buffer, and 1 division operation). Compared with the multiplication calculation process shown in FIG. 3, there are 4 fewer calculations with the amplification factor SF. That is, assuming the number of blocks N, before optimization (i.e., in the process shown in FIG. 3), the number of calculations in which the amplification factor SF participates is equal to 2xN+1. After optimization (i.e., in the process shown in FIG. 5), the number of calculations in which the amplification factor SF participates is equal to 2x(N / N L1 )+1. After optimization, the number of calculations is reduced by 2xN+1-(2x(N / N L1 )+1)=2xN(1-1 / N L1 ). It can be understood that the first storage number N L1 in the L1 buffer is usually large, so the number of calculations in which the amplification factor SF participates will be significantly reduced by the process shown in FIG. 5, thereby ensuring the accuracy of matrix multiplication in parallel computing hardware while effectively improving the running efficiency of matrix multiplication.

[0136] Referring to FIG. 6, it is an algorithm flowchart of the multiplication operation of a half-precision matrix and an error matrix provided by the embodiment of the application. As shown in FIG. 6, it is an algorithm flowchart of the multiplication operation A half xΔB half of a first single-precision matrix and a second error matrix, which specifically refers to the related flow description of steps 100 to 300, i.e., it refers to the above-mentioned use of the L1 buffer as a reuse intermediate cache and the setting of the first sub-block matrix The dual-buffer location effectively ensures the accuracy of matrix multiplication while improving its efficiency in parallel computing hardware.

[0137] Furthermore, because instructions for different functional units are typically distributed across different pipelines in AI chips (such as the Ascend processor), the correctness of timing relationships between these pipelines cannot be automatically guaranteed. Even within the same pipeline, the Ascend processor cannot guarantee dependencies between them. The following lists the pipelines used in matrix multiplication.

[0138] Referring to Figure 7, it is a schematic diagram of a pipeline configuration provided in an embodiment of this application. As shown in Figure 7, the pipeline name for performing vector operations on source data is PIPE_V by default, the pipeline name for performing matrix operations on source data is PIPE_M by default, the pipeline name for transferring source data from the unified buffer UB to the global buffer GM or the L1 buffer is PIPE_MTE3 by default, the pipeline name for transferring source data from the unified buffer UB to the unified buffer UB or the L0C memory unit is PIPE_V by default, and so on.

[0139] In commonly used AI chips (such as the Ascend hardware platform), manual pipelined operation is disabled by default, meaning automatic pipelined operation is used, with the hardware handling the execution dependencies of each computational unit. When implementing the algorithm flow shown in Figure 6, although automatic pipelined operation can ensure the correctness of timing relationships, it cannot achieve optimal performance, which will seriously affect the overall performance of matrix multiplication.

[0140] Therefore, in this embodiment, by manually specifying the dependencies between the data transport and data computation pipelines, the pipeline dependencies are simplified to ensure the correctness of the timing relationships, thereby improving the performance of matrix multiplication in parallel computing hardware.

[0141] Among them, before step 100: obtaining the first half-precision matrix of the first single-precision matrix and the second half-precision matrix of the second single-precision matrix, the optimization method for implementing high-performance single-precision matrix multiplication based on the Ascend half-precision computing unit also includes the step of: pipeline locking the block error module.

[0142] After obtaining multiple second block error matrices from the second error matrix and storing them in the L1 buffer of the parallel computing hardware in step 200, the optimization method for high-performance single-precision matrix multiplication based on the Ascend half-precision computing unit further includes the step of: pipeline unlocking the block error module.

[0143] In step 300: before obtaining the first sub-block matrix in the first half-precision matrix one by one and storing it in the L1 buffer, the optimization method for implementing high-performance single-precision matrix multiplication by the Ascend-based half-precision calculation unit further includes the step of: pipeline locking the sub-block calculation module.

[0144] In addition, after step 300: obtaining the first result matrix of the first half-precision matrix and the second error matrix based on the first sub-block result matrix, the optimization method for implementing high-performance single-precision matrix multiplication by the Ascend-based half-precision calculation unit further includes the step of: pipeline unlocking the sub-block calculation module.

[0145] In some embodiments, the sub-block error module includes the generation and storage of the second sub-block error matrix to the L1 buffer, that is, in the algorithm flow as shown in FIG. 6, the corresponding for loop in the process of obtaining the second sub-block error matrix and storing it to the L1 buffer is abstracted as an MTE3 pipeline (i.e., the sub-block error module). It can be understood that this process can be abstracted as an MTE3 pipeline because the purpose of this for loop is to obtain the second sub-block error matrix and store it to the L1 buffer, which is exactly an PIPE_MTE3 pipeline; in addition, in the algorithm flow as shown in FIG. 6, the PIPE_MTE3 pipeline is not involved in the subsequent use of the L1 buffer. This example is only a simple example, and different abstraction methods will have different effects on performance when abstracting multiple data transfer and calculation.

[0146] In addition, the sub-block calculation module includes the storage of the first sub-block matrix to the L1 buffer and the matrix calculation process of the first sub-block matrix and the second sub-block error matrix, that is, in the algorithm flow as shown in FIG. 6, the process of generating the first sub-block matrix , storing it to the L1 buffer, and multiplying it with the corresponding second sub-block error matrix is abstracted as an MTE1 pipeline (i.e., the sub-block calculation module).

[0147] In this embodiment, the sub-block error module and the sub-block calculation module are separated from each other, so as to avoid the situation that the operation of is affected due to the change of the second sub-block error matrix in one pipeline, thereby effectively improving the correctness of the timing relationship and improving the accurate performance of matrix multiplication in parallel operation hardware.

[0148] The embodiment of the present application abstracts the plurality of for loops into two pipeline modules (including a block error module and a block calculation module) in the algorithm flow as shown in FIG. 6. Such abstraction has two benefits: on the one hand, the dependency relationship between the pipelines involving the for loops is simplified. On the other hand, the for loop is abstracted into a pipeline, and the for loop does not involve the external pipeline, and the pipeline dependency relationship in the for loop is also simplified. By abstracting the for loop into a pipeline, the pipeline relationship inside and outside the for loop is simplified, which accelerates the operation speed of the matrix multiplication.

[0149] Referring to FIG. 8, it is another algorithm flow diagram of the half-precision matrix and error matrix multiplication provided by the embodiment of the present application. As shown in FIG. 8, it is an algorithm flow generated based on the algorithm flow as shown in FIG. 6 and in combination with the manually specified data carrying and data calculation pipeline dependency relationship, that is, the first dashed box in FIG. 8 is taken as a block error module, and in the process of the block error module, the critical section between the pipeline MTE3 corresponding to the block error module and the pipeline MTE1 corresponding to the block calculation module is locked until all function operations corresponding to the block error module are executed; and then the critical section between the pipeline MTE3 and the pipeline MTE1 is unlocked. In addition, the second dashed box in FIG. 8 is taken as a block calculation module, and when the block calculation module is executed, similar to the execution of the block error module, the critical section between the pipeline MTE3 and the pipeline MTE1 is also locked until all function operations in the block calculation module are executed.

[0150] Referring to FIG. 9, it is a flow diagram of a double buffer memory switching provided by the embodiment of the present application. Based on the switching of the critical sections of the plurality of modules in the algorithm flow generated in combination with the manually specified data carrying and data calculation pipeline dependency relationship, the advantages of the double buffer mechanism set in the L1 buffer are further described. As shown in FIG. 9, A half × ΔB half Taking four blocks as an example, the double buffer involves two blocks of memory (two blocks are generic, and in fact all variables participating in the calculation are allocated two non-overlapping memories), and one is responsible for calculation and one is responsible for data carrying at the same time. Therefore, special processing is required when manually specifying the pipeline, and the following scheme is used to optimize the double buffer pipeline: (1) all pipelines involved in the double buffer are abstracted into an M pipeline (that is, the second gray part in the block calculation module as shown in FIG. 8), which simplifies the dependency relationship between the double buffer pipeline and other pipelines. (2) Two critical sections are applied, and the two blocks of memory used by the double buffer are protected by different critical sections, and the two critical sections alternately protect the pipelines involved in the double buffer, thereby effectively ensuring the accuracy of the matrix multiplication in the parallel operation hardware while improving the running efficiency of the matrix multiplication.

[0151] wherein the accumulating each first partial result matrix corresponding to each first partial matrix to obtain the first result matrix of the first half-precision matrix and the second error matrix comprises the following step 350.

[0152] Step 350: Accumulating each first partial result matrix corresponding to each first partial matrix to obtain the first result matrix of the first half-precision matrix and the second error matrix.

[0153] The following Step 350 is described in detail.

[0154] In some embodiments, in the operation process of A half × ΔB half , after obtaining each first partial result matrix corresponding to each first partial matrix and the corresponding second error partial matrix , the first result matrix obtained by matrix operation of the first half-precision matrix A half and the second error matrix ΔB half can be obtained by accumulating all first partial result matrices.

[0155] The above is an example of the multiplication operation of the first half-precision matrix and the second error matrix A half × ΔB half , and the processing flow of the multiplication operation of the rest of the first half-precision matrix and the second half-precision matrix A half × B half , and the multiplication operation of the first error matrix and the second half-precision matrix ΔA half × B half is similar.

[0156] Step 400: Obtaining the first error matrix between the first single-precision matrix and the first half-precision matrix.

[0157] The following describes step 400 in detail.

[0158] In some embodiments, in addition to performing the multiplication operation of the first half-precision matrix and the second error matrix A half × ΔB half , the processing flow of the multiplication operation of the first error matrix and the second half-precision matrix ΔA half × B half also needs to be performed, and similar to the above obtaining the second error matrix ΔB half , after the first single-precision matrix A single is converted into the multiplication of the first half-precision matrix A half , the first single-precision matrix Asingle Subtracting the first half-precision matrix A half The first single-precision error matrix ΔA obtained after the single-precision conversion single = A single -to-single(A half ), and then performing half-precision conversion on the first single-precision error matrix to obtain the first error matrix ΔA half , in the case that the first single-precision matrix B single is too large, the first error matrix ΔA half includes a plurality of first sub-block error matrices

[0159] Step 500: obtaining a second result matrix obtained by matrix multiplication of the first error matrix and the second half-precision matrix, obtaining a third result matrix obtained by matrix multiplication of the first half-precision matrix and the second half-precision matrix, accumulating the first result matrix, the second result matrix and the third result matrix to obtain a single-precision target matrix, and taking the single-precision target matrix as the result of the matrix multiplication of the first single-precision matrix and the second single-precision matrix.

[0160] The step 500 is described in detail as follows.

[0161] In some embodiments, in addition to performing the multiplication operation A half × ΔB half part, and after obtaining the first error matrix, it is also necessary to obtain a second result matrix obtained by matrix multiplication of the first error matrix and the second half-precision matrix ΔA half × B half and a third result matrix obtained by matrix multiplication of the first half-precision matrix and the second half-precision matrix A half × B half , similar to the steps 200 to 300 described above. Finally, the first result matrix, the second result matrix and the third result matrix are accumulated to obtain a half-precision target matrix C half , the half-precision target matrix is then converted into a single-precision target matrix C single , and the single-precision target matrix is taken as the result of the matrix multiplication of the first single-precision matrix and the second single-precision matrix.

[0162] Wherein, when the operations of A half × ΔB half part and ΔA half × B half part are performed in sequence for the same AI chip, since A half × ΔB halfDuring some operations, the amount of space planned in the L1 buffer of the AI ​​chip is based on the second block half-precision matrix. The size of the matrix b k b n The setup was intended to accommodate more second-block half-precision matrices. However, after performing ΔA... half ×B half During partial computation, the first block error matrix The size of the matrix b m b k The configuration is incompatible and therefore cannot be used directly; reconfiguration is required. However, reconfiguration will further increase computation time costs. Therefore, in this embodiment, ΔA is performed later. half ×B half During partial computation, the first block of half-precision matrix is ​​processed according to the size of the pre-defined L1 buffer. Matrix size and second block matrix The matrix size is adaptively adjusted to improve computation speed. This is described in detail below.

[0163] The first result matrix, which is obtained based on the first block result matrix, includes the following steps 510 to 540: obtaining the first half-precision matrix and the second error matrix.

[0164] Step 510: Obtain the size of the third block based on the size of the first block, and obtain the size of the fourth block based on the size of the second block.

[0165] Step 520: Split the second half-precision matrix according to the third block size to obtain multiple second block matrices.

[0166] Step 530: Split the first error matrix according to the size of the fourth block to obtain multiple first block error matrices.

[0167] Step 540: Obtain the second block result matrix by performing matrix multiplication of each second block matrix and the first block error matrix through the L1 buffer and the matrix calculation module, and obtain the second result matrix based on all the second block result matrices.

[0168] Steps 510 to 540 are described in detail below.

[0169] In some embodiments, when A is executed half ×ΔB half After partial calculations, proceed with ΔA. half ×B half During partial computation, in order to directly combine multiple first block error matrices Directly put into the L1 buffer, based on the original first block matrix First block size b m b k Get the size of the third block b k b m And based on the original second error block matrix The second block size b k b n The size of the fourth block, b, is obtained. n b k .

[0170] Next, based on the size b of the third block k b m For the second half-precision matrix B half The process involves splitting the matrix into multiple second-part matrices. And according to the size b of the fourth block k b n For the first error matrix ΔA half The data is split to obtain multiple first block error matrices. It is understandable that the second block matrix at this time... The size is the third block size b k b m The first block error matrix The size is the fourth block size b k b n The initial storage quantity N at this point L1 The formula will become N L1 =(1024-2b) k b m ) / (b n b k Due to the block size b m b k b n It can be dynamically adjusted, reusing the same set of algorithm processing logic and code, facilitating optimization that takes effect on each module. The L1 buffer layout does not need to be changed, making it applicable to more scenarios, while the memory layout of UB does not need to be altered, effectively improving the computation speed in parallel computing hardware.

[0171] Then, similar to steps 200 to 300 above, each second block matrix and the first block error matrix are obtained by matrix multiplication through the L1 buffer and the matrix calculation module to obtain the second block result matrix, and the second result matrix is ​​obtained based on all the second block result matrices.

[0172] To further verify the reliability of the optimization method for implementing high-performance single-precision matrix multiplication by the half-precision computing unit based on the Ascend provided in the application, the embodiments of the application also perform simulation performance verification on the optimization method for implementing high-performance single-precision matrix multiplication by the half-precision computing unit based on the Ascend.

[0173] In some embodiments, the block size and the efficiency of data transfer can affect the performance of matrix multiplication calculation. The capacity of the unified buffer UB cache (256 KB) and the L1 buffer (1024 KB) limits the block size and the number of data stored in the L1 buffer, which cannot be infinite. According to the hardware requirements of the Ascend processor: 1, the CUBE computing unit requires that the block size must be a multiple of 16, and 2, the capacity limit of the L0C is 256 KB. Under the above hardware constraints, in order to verify the performance improvement of the matrix multiplication provided in the application and verify how high the performance can be achieved, a large block is selected as much as possible, and the efficiency of the L1 buffer (i.e., the number of FP16 data that the L1 buffer can store) is maximized. The following data points are selected for testing. In the test, all 32 AI chips in the parallel running hardware are used, and the input size of the test data is M = AI chip number x b m x NUM, N = AI chip number x b n x NUM, K = b k x 2 x N L1 x NUM, wherein: NUM is to use as large an input as possible for testing, according to the limit of the global buffer GM = 1 GB, NUM = 6 in this test.

[0174] Referring to FIG. 10, it is a parameter diagram for performance simulation of a data processing method provided in an embodiment of the application. As shown in FIG. 10, b m , b k , and b n are the sizes of the matrix blocks, N L1 is the maximum number of blocks that can be placed in the L1 cache at a time, and the L1 usage capacity is the number of FP16 data stored in the L1 cache.

[0175] To verify the effect, the evaluation formula used is: relative residual = (||C FP64 -C NPU || F ) / ||C FP64 || F .

[0176] Wherein ||·|| F represents the L2 norm, C FP64 is the result of the cblas_dgemm calculation of the OpenBLAS library, and C NPUis the calculation result on the Ascend 910A processor. The actual effect of the optimization of the patent is verified from three directions as follows.

[0177] Referring to FIG. 11, it is a performance simulation diagram of the first data processing method provided by the embodiment of the application. As shown in FIG. 11, by comparing the accuracy with the running result of the BLAS library, it can be found that when the block calculation matrix multiplication is used, the optimization performance scheme used by the application is basically consistent with the sgemm of the BLAS library in the matrix multiplication accuracy. When the sgemm function of the BLAS library does not perform block, the optimization performance scheme used by the patent is more optimal in accuracy. This shows that the optimization algorithm of the application can meet the accuracy requirement of single-precision matrix multiplication.

[0178] Referring to FIG. 12, it is a performance simulation diagram of the second data processing method provided by the embodiment of the application. As shown in FIG. 12, by continuously adding the optimization scheme in the optimization algorithm of the application, it can be observed that the optimization scheme improves the performance efficiency of the single-precision matrix multiplication operation. When no optimization scheme is added, the average performance of the test is 3TFlops. After adding the reuse L1 and double cache optimization scheme, the average performance of the test is 15.08TFlops, and the performance is improved by about 5 times. After further adding the manual assignment pipeline optimization scheme, the average performance of the test is 52.85TFlops, and the performance is improved by about 3.5 times. Through the optimization scheme of the patent, the average performance is improved from 3TFlops to 52.85TFlops, and the overall performance is improved by about 17.6 times, and the performance is significantly improved.

[0179] Referring to FIG. 13, it is a performance simulation diagram of the third data processing method provided by the embodiment of the application. As shown in FIG. 13, the single-precision matrix multiplication operation is simulated by the CUBE operation unit through FP16 on the 910A hardware that does not support single-precision matrix multiplication. Before the optimization scheme of the application is executed, the optimal performance is 4.64TFlops; after the optimization scheme proposed by the application is executed, the optimal performance is 62.36TFlops, which is improved by nearly 12.4 times. At the same time, through the actual effect analysis, the optimal block combination of the optimization scheme proposed by the patent is (b m = 192, b k = 80, b n = 192, N L = 32), and the actual performance effect reaches 73% of the equivalent single-precision matrix multiplication peak value (256TFlops / 3 = 85.33TFlops) on the Ascend 910A. On the hardware that does not support single-precision matrix multiplication, the performance is significantly improved.

[0180] The optimization method for high-performance single-precision matrix multiplication implemented by the half-precision calculation unit based on Ascend and the related equipment are provided in the embodiments of the present application. The method comprises the following steps: firstly, the blocking error module is locked in pipeline, a first half-precision matrix of a first single-precision matrix and a second half-precision matrix of a second single-precision matrix are obtained; then, a second error matrix between the second single-precision matrix and the second half-precision matrix is obtained, the error matrix storage capacity is obtained based on the difference between the cache capacity of the L1 buffer and the double quantity of the first blocking size, the first storage quantity is obtained based on the ratio of the error matrix storage capacity to the second blocking size, the second blocking error matrix corresponding to the first storage quantity is stored in the L1 buffer, the blocking error module is unlocked in pipeline, and the second error matrix is a half-precision matrix; thereafter, the blocking calculation module is locked in pipeline, a second first blocking matrix is obtained according to the arrangement order of a plurality of first blocking matrices in the first half-precision matrix and is stored in a first storage position, a plurality of second blocking error matrices in the L1 buffer are multiplied by a magnification factor to obtain magnified second blocking error matrices, the magnified second blocking error matrices corresponding to the first blocking matrix in the L1 buffer are transmitted to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first blocking result matrix, the first blocking result matrix is obtained, and the first blocking result matrix corresponding to each first blocking matrix is accumulated to obtain a first result matrix of the first half-precision matrix and the second error matrix, the blocking calculation module is unlocked in pipeline; and a first error matrix between the first single-precision matrix and the first half-precision matrix is obtained, and the first error matrix is a half-precision matrix; finally, a third blocking size is obtained according to the first blocking size, a fourth blocking size is obtained according to the second blocking size, the second half-precision matrix is split according to the third blocking size to obtain a plurality of second blocking matrices, the first error matrix is split according to the fourth blocking size to obtain a plurality of first blocking error matrices, each second blocking matrix and first blocking error matrix are sequentially obtained to perform matrix multiplication through the L1 buffer and the matrix calculation module to obtain a second blocking result matrix, a second result matrix is obtained based on all second blocking result matrices, a third result matrix is obtained by performing matrix multiplication on the first half-precision matrix and the second half-precision matrix, the single-precision target matrix is obtained by accumulating the first result matrix, the second result matrix and the third result matrix, and the single-precision target matrix is taken as the result of the matrix multiplication of the first single-precision matrix and the second single-precision matrix.

[0181] The embodiments of the present application can effectively reduce the pressure on the use of the unified buffer UB by using the L1 buffer as the intermediate cache for multiplexing in the multiplication operation of the block matrix, and can divide larger matrix blocks when performing matrix division by using the larger storage space of the L1 buffer, so that the data carrying frequency and the number of calls to the matrix calculation unit CUBE are reduced, thereby facilitating the acceleration of the matrix multiplication rate. Moreover, by using the L1 buffer as the intermediate cache for multiplexing in the multiplication operation of the block matrix, more data can be stored at one time, that is, most of the storage space is used to pre-store the second error matrix, so that the number of times of cross execution of data carrying and data calculation is reduced, which is more conducive to the performance of matrix multiplication, thereby effectively improving the calculation speed of the matrix multiplication operation of the first half-precision matrix and the second error matrix, to improve the calculation rate of the matrix operation in the parallel computing hardware; and by setting a double buffer area in the L1 buffer, the data carrying and data calculation are synchronized at the same time, that is, the algorithm row in the black box in FIG. 4 can be implemented at the same time, thereby effectively improving the calculation efficiency of the matrix multiplication in the parallel computing unit; and by the flow as shown in FIG. 5, the calculation of the scaling factor SF for multiple matrices in the L1 buffer is performed together, so that the number of times of participation in the calculation of the scaling factor SF is obviously reduced, thereby ensuring the accuracy of the matrix multiplication in the parallel operation hardware while effectively improving the operation efficiency of the matrix multiplication; in addition, by abstracting multiple for loops into two pipeline modules (including a block error module and a block calculation module) in the algorithm flow as shown in FIG. 6, the dependence relationship between the pipelines involving the for loops is simplified, and since one for loop is abstracted into one pipeline, the for loop does not involve external pipelines, and the dependence relationship between the internal pipelines of the for loop is also simplified. By abstracting the for loop into a pipeline, the relationship between the internal and external pipelines is simplified, which accelerates the matrix multiplication speed; finally, since the block size b m 、b k 、b n can be dynamically adjusted, the same set of algorithm processing logic and code can be reused, which is convenient for optimization to take effect on each module. The arrangement of the L1 buffer does not need to be changed, which can be applied to more scenarios, and the memory layout of the UB does not need to be changed, thereby effectively ensuring the reliability of the matrix multiplication operation while improving the calculation rate of the matrix operation in the parallel computing hardware.

[0182] The embodiments of the present application also provide a data processing apparatus for implementing high-performance single-precision matrix multiplication based on an Ascend half-precision calculation unit, which can implement the optimization method for implementing high-performance single-precision matrix multiplication based on the Ascend half-precision calculation unit. Referring to FIG. 14, the apparatus 1400 includes:

[0183] The matrix obtaining module 1410 is configured to obtain a first half-precision matrix of a first single-precision matrix and a second half-precision matrix of a second single-precision matrix.

[0184] The first error matrix module 1420 is configured to obtain a second error matrix between the second single-precision matrix and the second half-precision matrix, obtain a plurality of second block error matrices in the second error matrix, and store the plurality of second block error matrices in an L1 buffer of the parallel computing hardware, the second error matrix being a half-precision matrix.

[0185] The matrix calculation module 1430 is configured to obtain a first block matrix in the first half-precision matrix one by one, store the first block matrix in the L1 buffer, and transmit the first block matrix and a second block error matrix in the L1 buffer to a matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first block result matrix, and obtain a first result matrix of the first half-precision matrix and the second error matrix based on the first block result matrix.

[0186] The second error matrix module 1440 is configured to obtain a first error matrix between the first single-precision matrix and the first half-precision matrix, the first error matrix being a half-precision matrix.

[0187] The result calculation module 1450 is configured to obtain a second result matrix obtained by matrix multiplication of the first error matrix and the second half-precision matrix, obtain a third result matrix obtained by matrix multiplication of the first half-precision matrix and the second half-precision matrix, accumulate the first result matrix, the second result matrix, and the third result matrix to obtain a single-precision target matrix, and take the single-precision target matrix as a result of matrix multiplication of the first single-precision matrix and the second single-precision matrix.

[0188] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment are basically the same as the specific embodiments of the data processing apparatus for implementing high-performance single-precision matrix multiplication based on the Ascend half-precision computing unit and the specific embodiments of the optimization method for implementing high-performance single-precision matrix multiplication based on the Ascend half-precision computing unit. Here, the details are not repeated.

[0189] In the embodiment of the present application, the high-performance single-precision matrix multiplication data processing device based on the semi-precision calculation unit of Ascend can effectively reduce the pressure on the use of the unified buffer UB by using the L1 buffer as the intermediate cache for multiplexing in the multiplication operation of the block matrix, and can divide larger matrix blocks when performing matrix division by using the larger storage space of the L1 buffer, so that the number of data transfer and the number of CUBE matrix calculation unit calls is reduced, thereby facilitating the acceleration of the matrix multiplication rate. Moreover, by using the L1 buffer as the intermediate cache for multiplexing in the multiplication operation of the block matrix, more data can be stored at one time, i.e., most of the storage space is used to pre-store the second error matrix, so that the number of cross-executions of data transfer and data calculation is reduced, which is more conducive to the performance of matrix multiplication, thereby effectively improving the calculation speed of the matrix multiplication of the first semi-precision matrix and the second error matrix, and improving the calculation rate of matrix operation in parallel computing hardware; and by setting a double buffer area in the L1 buffer, the data transfer and data calculation are synchronized at the same time, i.e., the algorithm row in the black box in FIG. 4 can be implemented at the same time, thereby effectively improving the calculation efficiency of matrix multiplication in the parallel computing unit; and by performing the calculation of the scale factor SF on multiple matrices in the L1 buffer as shown in FIG. 5, the number of times the scale factor SF participates in the calculation can be significantly reduced, thereby ensuring the accuracy of matrix multiplication in parallel operation hardware while effectively improving the operation efficiency of matrix multiplication. In addition, by abstracting multiple for loops into two pipeline modules (including a block error module and a block calculation module) in the algorithm process shown in FIG. 6, the dependence relationship between the pipelines involving for loops is simplified, and since one for loop is abstracted into a pipeline, the for loop does not involve external pipelines, and the pipeline dependence relationship in the for loop is also simplified. By abstracting the for loop into a pipeline, the relationship between the internal and external pipelines is simplified, which accelerates the matrix multiplication speed; finally, since the block size b m 、b k 、b n can be dynamically adjusted, the same set of algorithm processing logic and code can be reused, which is convenient for optimization of each module. The arrangement of the L1 buffer does not need to be changed, which can be applied to more scenarios, and the memory layout of the UB does not need to be changed, thereby effectively ensuring the reliability of matrix multiplication operation while improving the calculation rate of matrix operation in parallel computing hardware.

[0190] The embodiment of the present application also provides an electronic device, comprising:

[0191] at least one memory;

[0192] at least one processor;

[0193] at least one program;

[0194] The program is stored in the memory, and the processor executes the at least one program to implement the optimization method of the Ascend-based half-precision calculation unit to realize high-performance single-precision matrix multiplication. The electronic device can be any smart terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.

[0195] Please refer to FIG. 15, which shows the hardware structure of the electronic device of another embodiment, including:

[0196] The processor 1501 can be implemented in the form of a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided by the embodiments of the present application;

[0197] The memory 1502 can be implemented in the form of a ROM (Read Only Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory), etc. The memory 1502 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are saved in the memory 1502 and are called and executed by the processor 1501 to implement the optimization method of the Ascend-based half-precision calculation unit to realize high-performance single-precision matrix multiplication;

[0198] The input / output interface 1503 is used to realize information input and output;

[0199] The communication interface 1504 is used to realize the communication interaction between the device and other devices. The communication can be realized by wired means (such as USB, network cable, etc.) or by wireless means (such as mobile network, WIFI, Bluetooth, etc.);

[0200] The bus 1505 transmits information between the various components of the device (such as the processor 1501, the memory 1502, the input / output interface 1503, and the communication interface 1504);

[0201] The processor 1501, the memory 1502, the input / output interface 1503, and the communication interface 1504 are communicatively connected to each other within the device through the bus 1505.

[0202] The embodiment of the present application also provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the optimization method for realizing high-performance single-precision matrix multiplication by an Ascend-based half-precision calculation unit.

[0203] The memory is a non-transitory computer readable storage medium, and can be used to store a non-transitory software program and a non-transitory computer executable program. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, for example, at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and the remote memory can be connected to the processor through a network. Examples of the network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0204] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0205] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than the figures, or combine certain steps, or different steps.

[0206] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, that is, can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the modules can be selected to achieve the purpose of the embodiments of the present application.

[0207] Those skilled in the art can understand that all or some of the steps in the above disclosed method, the function modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0208] The terms "first", "second", "third", "fourth", and the like in the description of this application and in the claims hereof, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed herein is solely for the convenience of the reader and does not limit the scope of the application. It is also to be understood that the description and examples in this application are intended to cover all possible combinations where any of the several elements can represent one or more elements.

[0209] It should be understood that, in the application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0210] In several embodiments provided in the application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the above-mentioned units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0211] The units described above as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment of the application.

[0212] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0213] When the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions used to cause a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various other media that can store programs.

[0214] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not intended to limit the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

An optimization method for high-performance single-precision matrix multiplication based on a semi-precision computing unit of Ascend, characterized by, The method comprises: obtaining a first half-precision matrix of a first single-precision matrix and a second half-precision matrix of a second single-precision matrix; obtaining a second error matrix between the second single-precision matrix and the second half-precision matrix, obtaining a plurality of second block error matrices in the second error matrix, and storing the second block error matrices in an L1 buffer of parallel computing hardware, the second error matrix being a half-precision matrix; obtaining a first block matrix in the first half-precision matrix one by one, storing the first block matrix in the L1 buffer, transmitting the first block matrix and the second block error matrix in the L1 buffer to a matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first block result matrix, and obtaining a first result matrix of the first half-precision matrix and the second error matrix based on the first block result matrix; obtaining a first error matrix between the first single-precision matrix and the first half-precision matrix, the first error matrix being a half-precision matrix; obtaining a second result matrix obtained by matrix multiplication of the first error matrix and the second half-precision matrix, obtaining a third result matrix obtained by matrix multiplication of the first half-precision matrix and the second half-precision matrix, accumulating the first result matrix, the second result matrix, and the third result matrix to obtain a single-precision target matrix, and taking the single-precision target matrix as a result of matrix multiplication of the first single-precision matrix and the second single-precision matrix. The method of claim 1, wherein The obtaining of the plurality of second block error matrices in the second error matrix and the storing of the second block error matrices in the L1 buffer of the parallel computing hardware comprises: determining a first storage quantity based on a first block size of the first block matrix and a second block size of the second block error matrix; storing the second block error matrix corresponding to the first storage quantity in the L1 buffer. The method according to claim 2, characterized in that The determination of the first storage quantity based on the first block size of the first block matrix and the second block size of the second block error matrix comprises: obtaining an error matrix storage capacity based on a difference between a cache capacity of the L1 buffer and twice the first block size; obtaining the first storage quantity based on a ratio of the error matrix storage capacity to the second block size. The method according to claim 3, characterized in that The L1 buffer is provided with a first storage location and a second storage location for storing the first block matrix, and the obtaining of the first block matrix in the first half-precision matrix one by one, the storing of the first block matrix in the L1 buffer, the transmitting of the first block matrix and the second block error matrix in the L1 buffer to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first block result matrix comprises: obtaining a second first block matrix according to an arrangement order of a plurality of the first block matrices in the first half-precision matrix and storing the second first block matrix in the first storage location; transmitting the first first-block matrix in the second storage location and the second second-block error matrix corresponding to the first first-block matrix in the L1 buffer to a matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first first-block result matrix. The method of claim 1, wherein The method further includes: amplifying the second second-block error matrices in the L1 buffer to obtain amplified second second-block error matrices; transmitting the amplified second second-block error matrix corresponding to the first first-block matrix in the L1 buffer to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain the first first-block result matrix. The method according to claim 2, characterized in that The method further includes: locking a pipeline of a first-block error module, the first-block error module including generation and storage of the second second-block error matrix in the L1 buffer; The method further includes: unlocking the pipeline of the first-block error module. The method according to claim 2, characterized in that The method further includes: locking a pipeline of a first-block calculation module, the first-block calculation module including storage of the first first-block matrix in the L1 buffer and matrix calculation of the first first-block matrix and the second second-block error matrix; The method further includes: unlocking the pipeline of the first-block calculation module. The method according to claim 4, characterized in that The method further includes: accumulating the first first-block result matrix corresponding to each of the first first-block matrices to obtain the first half-precision matrix and the first result matrix of the second error matrix. The method according to claim 2, characterized in that The method further includes: obtaining a third first-block size according to the first first-block size and a fourth first-block size according to the second second-block size; splitting the second half-precision matrix according to the third first-block size to obtain a plurality of second second-block matrices; splitting the first error matrix according to the fourth first-block size to obtain a plurality of first first-block error matrices; obtaining a second second-block result matrix of each of the second second-block matrices and the first first-block error matrix through the L1 buffer and the matrix calculation module, and obtaining the second result matrix based on all the second second-block result matrices. A data processing apparatus based on an Ascend half-precision computing unit implements high-performance single-precision matrix multiplication, characterized in that, The apparatus includes: The matrix obtaining module is configured to obtain a first half-precision matrix of a first single-precision matrix and a second half-precision matrix of a second single-precision matrix. The first error matrix module is configured to obtain a second error matrix between the second single-precision matrix and the second half-precision matrix, obtain a plurality of second sub-block error matrices in the second error matrix, and store the second error matrix in an L1 buffer of parallel computing hardware, the second error matrix being a half-precision matrix. The matrix calculation module is configured to obtain a first sub-block matrix in the first half-precision matrix one by one, store the first sub-block matrix in the L1 buffer, and transmit the first sub-block matrix in the L1 buffer and the second sub-block error matrix to a matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first sub-block result matrix, and obtain a first result matrix of the first half-precision matrix and the second error matrix based on the first sub-block result matrix. The second error matrix module is configured to obtain a first error matrix between the first single-precision matrix and the first half-precision matrix, the first error matrix being a half-precision matrix. The result calculation module is configured to obtain a second result matrix obtained by multiplying the first error matrix and the second half-precision matrix, obtain a third result matrix obtained by multiplying the first half-precision matrix and the second half-precision matrix, accumulate the first result matrix, the second result matrix, and the third result matrix to obtain a single-precision target matrix, and take the single-precision target matrix as a result of matrix multiplication of the first single-precision matrix and the second single-precision matrix. The computer program is executed by the processor to implement the optimization method for high-performance single-precision matrix multiplication of the Ascend-based half-precision calculation unit according to any one of claims 1 to 9. An electronic device, characterized by comprising: The computer program is executed by the processor to implement the optimization method for high-performance single-precision matrix multiplication of the Ascend-based half-precision calculation unit according to any one of claims 1 to 9. A computer-readable storage medium having stored thereon a computer program, characterized in that ​

Images