Data processing method for matrix operation in parallel computing hardware and related equipment

Abstract

The embodiment of the invention provides a data processing method for matrix operation in parallel computing hardware and related equipment. Obtaining a first half-precision matrix of the first single-precision matrix, a second half-precision matrix of the second single-precision matrix, a second error matrix between the second single-precision matrix and the second half-precision matrix, and a plurality of second block error matrixes in the second error matrix, and storing the first half-precision matrix, the second half-precision matrix, the second error matrix and the plurality of second block error matrixes in an L1 buffer area of parallel computing hardware; obtaining first block matrixes in the first half-precision matrix one by one, storing the first block matrixes in an L1 buffer area, further performing matrix operation to obtain a first result matrix, obtaining a second result matrix and a third result matrix, and accumulating the first result matrix, the second result matrix and the third result matrix to obtain a single-precision target matrix; and the single-precision target matrix is used as a result of matrix multiplication operation of the first single-precision matrix and the second single-precision matrix, so that the precision and the calculation rate of single-precision matrix multiplication are effectively improved.

Description

Technical Field

[0001] The present application relates to the field of network security technology, and in particular to a data processing method and related equipment for matrix operations in parallel computing hardware. Background Art

[0002] With the continuous development of neural networks, scientific computing and artificial intelligence, these disciplines have gradually been applied to life and work in recent years. As the input feature maps used in these fields become larger and larger, and the number of layers and coupling relationships of the models used become more and more complex, computing performance has become one of the key indicators in these fields. Among them, matrix multiplication operations are widely used in these disciplines. Therefore, improving the performance of matrix multiplication operations is crucial.

[0003] In the related art, for the multiplication operation of single-precision matrices, a computing chip that can support single-precision calculation is usually used to continue the calculation, but the cost of the computing chip that supports single-precision calculation is relatively high. In addition, when using a computing chip that only supports half-precision calculation, the single-precision matrix to be multiplied is usually converted into a half-precision matrix, and then the half-precision is multiplied, but the accuracy of the result obtained by this solution is low. Summary of the invention

[0004] The embodiments of the present application provide a data processing method and related equipment for matrix operations in parallel computing hardware, which can improve the accuracy of single-precision matrix multiplication operations when using a computing chip that supports half-precision calculations.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a data processing method for matrix operations in parallel computing hardware, the method comprising:

[0006] Obtain a first half-precision matrix of the first single-precision matrix and a second half-precision matrix of the second single-precision matrix;

[0007] Obtain a second error matrix between the second single-precision matrix and the second half-precision matrix, obtain a plurality of second block error matrices in the second error matrix, and store them in an L1 buffer of the parallel computing hardware, wherein the second error matrix is ​​a half-precision matrix;

[0008] Obtaining first block matrices in the first half-precision matrix one by one, storing them in the L1 buffer, transmitting the first block matrix and the second block error matrix in the L1 buffer to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first block result matrix, and obtaining a first result matrix of the first half-precision matrix and the second error matrix based on the first block result matrix;

[0009] Obtaining a first error matrix between the first single-precision matrix and the first half-precision matrix, where the first error matrix is ​​a half-precision matrix;

[0010] A second result matrix obtained by matrix multiplication of the first error matrix and the second half-precision matrix is ​​obtained, a third result matrix obtained by matrix multiplication of the first half-precision matrix and the second half-precision matrix is ​​obtained, the first result matrix, the second result matrix and the third result matrix are accumulated to obtain a single-precision target matrix, and the single-precision target matrix is ​​used as a result of matrix multiplication of the first single-precision matrix and the second single-precision matrix.

[0011] In some embodiments, obtaining a plurality of second block error matrices in the second error matrix and storing them in an L1 buffer of the parallel computing hardware comprises:

[0012] Determining a first storage quantity based on a first block size of the first block matrix and a second block size of the second block error matrix;

[0013] The second block error matrices corresponding to the first storage quantity are stored in the L1 buffer.

[0014] In some embodiments, determining the first storage quantity based on a first block size of the first block matrix and a second block size of the second block error matrix includes:

[0015] Obtaining an error matrix storage capacity based on a difference between a cache capacity of the L1 buffer and twice the size of the first block;

[0016] The first storage quantity is obtained based on the ratio of the error matrix storage capacity to the second block size.

[0017] In some embodiments, the L1 buffer is provided with a first storage location and a second storage location for storing the first block matrix, the first block matrix in the first half-precision matrix is ​​obtained one by one, stored in the L1 buffer, the first block matrix and the second block error matrix in the L1 buffer are transmitted to the matrix calculation module of the parallel computing hardware for matrix multiplication, and the first block result matrix is ​​obtained, including:

[0018] According to the arrangement order of the plurality of the first block matrices in the first half-precision matrix, obtaining a second first block matrix and storing it in the first storage location;

[0019] The first first block matrix at the second storage location and the second block error matrix corresponding to the first first block matrix in the L1 buffer are transmitted to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first block result matrix.

[0020] In some embodiments, the step of transmitting the first block matrix and the second block error matrix in the L1 buffer to a matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first block result matrix includes:

[0021] In the L1 buffer, a plurality of the second block error matrices are multiplied by an amplification factor to obtain an amplified second block error matrix;

[0022] The enlarged second block error matrix corresponding to the first block matrix in the L1 buffer is transmitted to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain the first block result matrix.

[0023] In some embodiments, before obtaining the first half-precision matrix of the first single-precision matrix and the second half-precision matrix of the second single-precision matrix, the method further includes:

[0024] Pipeline locking a block error module, wherein the block error module includes a process of generating and storing the second block error matrix in the L1 buffer;

[0025] After acquiring the plurality of second block error matrices in the second error matrix and storing them in the L1 buffer of the parallel computing hardware, the method further includes:

[0026] The block error module is pipeline unlocked.

[0027] In some embodiments, before the first block matrices in the first half-precision matrix are acquired one by one and stored in the L1 buffer, the method further includes:

[0028] Pipeline locking a block calculation module, wherein the block calculation module includes storing the first block matrix into the L1 buffer, and a matrix calculation process of the first block matrix and the second block error matrix;

[0029] After obtaining a first result matrix of the first half-precision matrix and the second error matrix based on the first block result matrix, the method further includes:

[0030] The block computing module is pipeline unlocked.

[0031] In some embodiments, obtaining a first result matrix of the first half-precision matrix and the second error matrix based on the first block result matrix includes:

[0032] The first block result matrices corresponding to each of the first block matrices are accumulated to obtain a first result matrix of the first half-precision matrix and the second error matrix.

[0033] In some embodiments, obtaining a second result matrix obtained by performing matrix multiplication of the first error matrix and the second half-precision matrix includes:

[0034] Obtaining a third block size according to the first block size, and obtaining a fourth block size according to the second block size;

[0035] Splitting the second half-precision matrix according to the third block size to obtain a plurality of second block matrices;

[0036] Splitting the first error matrix according to the fourth block size to obtain a plurality of first block error matrices;

[0037] A second block result matrix is ​​obtained by performing matrix multiplication of each second block matrix and the first block error matrix through the L1 buffer and the matrix calculation module one by one, and the second result matrix is ​​obtained based on all the second block result matrices.

[0038] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application proposes a data processing device for matrix operations in parallel computing hardware, the device comprising:

[0039] A matrix acquisition module, used for acquiring a first half-precision matrix of the first single-precision matrix and a second half-precision matrix of the second single-precision matrix;

[0040] a first error matrix module, configured to obtain a second error matrix between the second single-precision matrix and the second half-precision matrix, obtain a plurality of second block error matrices in the second error matrix, and store them in an L1 buffer of the parallel computing hardware, wherein the second error matrix is ​​a half-precision matrix;

[0041] a matrix calculation module, configured to obtain first block matrices in the first half-precision matrix one by one, store them in the L1 buffer, transmit the first block matrix and the second block error matrix in the L1 buffer to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first block result matrix, and obtain a first result matrix of the first half-precision matrix and the second error matrix based on the first block result matrix;

[0042] A second error matrix module, obtaining a first error matrix between the first single-precision matrix and the first half-precision matrix, where the first error matrix is ​​a half-precision matrix;

[0043] A result calculation module is used to obtain a second result matrix obtained by matrix multiplication of the first error matrix and the second half-precision matrix, obtain a third result matrix obtained by matrix multiplication of the first half-precision matrix and the second half-precision matrix, accumulate the first result matrix, the second result matrix and the third result matrix to obtain a single-precision target matrix, and use the single-precision target matrix as the result of matrix multiplication of the first single-precision matrix and the second single-precision matrix.

[0044] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the data processing method for matrix operations in parallel computing hardware as described in the first aspect.

[0045] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium, and the storage medium stores a computer program. When the computer program is executed by a processor, it implements the data processing method for matrix operations in the parallel computing hardware described in the first aspect above.

[0046] The data processing method and related equipment for matrix operations in parallel computing hardware proposed in the embodiment of the present application include: first, obtaining a first half-precision matrix of a first single-precision matrix and a second half-precision matrix of a second single-precision matrix; then, obtaining a second error matrix between the second single-precision matrix and the second half-precision matrix, obtaining multiple second block error matrices in the second error matrix, and storing them in an L1 buffer of the parallel computing hardware, where the second error matrix is ​​a half-precision matrix; then, obtaining first block matrices in the first half-precision matrix one by one, storing them in the L1 buffer, and transmitting the first block matrix and the second block error matrix in the L1 buffer to a matrix calculation module of the parallel computing hardware for matrix processing. The first block result matrix is ​​obtained by matrix multiplication, and the first result matrix of the first half-precision matrix and the second error matrix is ​​obtained based on the first block result matrix; and the first error matrix between the first single-precision matrix and the first half-precision matrix is ​​obtained, and the first error matrix is ​​a half-precision matrix; finally, the second result matrix obtained by matrix multiplication of the first error matrix and the second half-precision matrix is ​​obtained, and the third result matrix obtained by matrix multiplication of the first half-precision matrix and the second half-precision matrix is ​​obtained, the first result matrix, the second result matrix and the third result matrix are accumulated to obtain the single-precision target matrix, and the single-precision target matrix is ​​used as the result of matrix multiplication of the first single-precision matrix and the second single-precision matrix. The embodiment of the present application utilizes the error matrix generated after the single-precision matrix is ​​converted to a half-precision matrix in advance in the parallel computing hardware, and then utilizes the product result of the error matrix and the half-precision matrix to compensate for the accuracy of the single-precision matrix multiplication operation, thereby effectively improving the accuracy of single-precision matrix multiplication when using parallel computing hardware that supports half-precision calculations; in addition, the L1 buffer in the parallel computing hardware is used as an intermediate cache area, combined with the high capacity and low real-time transmission rate of the L1 buffer, so that a large number of block matrices can be transferred to the L1 buffer in advance, so that in the actual calculation process, the real-time data exchange and transmission of matrix data in the L1 buffer is effectively reduced, thereby effectively improving the calculation rate of matrix operations in the parallel computing hardware.

[0047] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a structural diagram of an AI chip provided in one embodiment of the present application.

[0049] Figure 2 It is a flowchart of a data processing method for matrix operations in parallel computing hardware provided by another embodiment of the present application.

[0050] Figure 3 This is a flowchart of a single-precision matrix multiplication provided by another embodiment of the present application.

[0051] Figure 4 This is a schematic diagram of an algorithm flow of double cache optimization provided by another embodiment of the present application.

[0052] Figure 5 It is a flowchart of a block operation of a single-precision matrix multiplication provided by another embodiment of the present application.

[0053] Figure 6 This is a schematic diagram of an algorithm flow for multiplication operation of a half-precision matrix and an error matrix provided in yet another embodiment of the present application.

[0054] Figure 7 This is a pipeline configuration diagram provided by another embodiment of the present application.

[0055] Figure 8 This is a flowchart of an algorithm for multiplication of another half-precision matrix and an error matrix provided in yet another embodiment of the present application.

[0056] Fig. 9 This is a schematic diagram of a dual-buffer memory switching process provided by another embodiment of the present application.

[0057] Fig.10 It is a parameter diagram of a performance simulation of a data processing method provided in yet another embodiment of the present application.

[0058] Fig.11 This is a performance simulation diagram of the first data processing method provided in yet another embodiment of the present application.

[0059] Fig.12 This is a performance simulation diagram of a second data processing method provided in yet another embodiment of the present application.

[0060] Fig.13 This is a performance simulation diagram of a third data processing method provided in yet another embodiment of the present application.

[0061] Fig.14 It is a structural diagram of a data processing device for matrix operations in parallel computing hardware provided in one embodiment of the present application.

[0062] Fig.15 It is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] It should be noted that although the functional modules are divided in the device schematic and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0066] First, some nouns involved in this application are analyzed:

[0067] Single-precision matrix multiplication refers to the operation of multiplying two matrices of single-precision floating-point type. Single-precision floating-point type usually uses 32-bit binary numbers to represent a floating-point number, of which 1 bit is used for the sign bit, 8 bits are used for the exponent, and 23 bits are used for the mantissa. Assuming that the size of matrix A is m×n and the size of matrix B is n×p, then the size of their product C is m×p, where the value of each element C[i][j] of matrix C is the sum of the products of the elements in the i-th row of matrix A and the elements in the j-th column of matrix B.

[0068] Similar to single-precision matrix multiplication, half-precision matrix multiplication refers to the multiplication of two matrices of half-precision floating-point type. Half-precision floating-point type usually uses 16 bits of binary to represent a floating-point number, of which 1 bit is used for the sign bit, 5 bits for the exponent, and 10 bits for the mantissa. Since the precision of half-precision floating-point type is lower, in practical applications, half-precision matrix multiplication is usually used in scenarios that require less precision, such as calculations in neural networks.

[0069] IEEE-754 is a floating-point representation of binary numbers. It is an international standard developed by the Institute of Electrical and Electronics Engineers (IEEE). This standard specifies the binary representation of floating-point numbers in computers, including the sign bit, exponent bit, and mantissa bit.

[0070] Parallel computing hardware refers to a hardware device that can perform multiple computing tasks simultaneously. These hardware devices usually have a high degree of parallel computing capabilities and efficient computing resource management capabilities, which can achieve efficient computing and data processing.

[0071] With the continuous development of neural networks, scientific computing and artificial intelligence, these disciplines have gradually been applied to life and work in recent years. As the input feature maps used in these fields become larger and larger, and the number of layers and coupling relationships of the models used become more and more complex, computing performance has become one of the key indicators in these fields. Among them, matrix multiplication operations are widely used in these disciplines. Therefore, improving the performance of matrix multiplication operations is crucial.

[0072] In the related art, for the multiplication operation of single-precision matrices, a computing chip that can support single-precision calculation is usually used to continue the calculation, but the cost of the computing chip that supports single-precision calculation is relatively high. In addition, when using a computing chip that only supports half-precision calculation, the single-precision matrix to be multiplied is usually converted into a half-precision matrix, and then the half-precision is multiplied, but the accuracy of the result obtained by this solution is low.

[0073] In order to improve the accuracy of single-precision matrix multiplication operations when using a computing chip that supports half-precision calculations, an embodiment of the present application uses an error matrix generated by converting a single-precision matrix into a half-precision matrix in advance in parallel computing hardware, and then uses the product of the error matrix and the half-precision matrix to compensate for the accuracy of the single-precision matrix multiplication process, thereby effectively improving the accuracy of single-precision matrix multiplication when using parallel computing hardware that supports half-precision calculations; in addition, the L1 buffer in the parallel computing hardware is used as an intermediate cache area, combined with the high capacity and low real-time transmission rate of the L1 buffer, so that a large number of block matrices can be transferred to the L1 buffer in advance, so that in the actual calculation process, the real-time data exchange and transmission of matrix data in the L1 buffer is effectively reduced, thereby effectively improving the calculation rate of matrix operations in the parallel computing hardware.

[0074] In order to better describe the data processing method for matrix operations in parallel computing hardware provided by the present application, the following first describes the parallel computing hardware used in the data processing method for matrix operations in parallel computing hardware. The parallel computing hardware includes multiple AI chips, which are connected to each other through a communication serial port. Figure 1 , is a schematic diagram of the structure of an AI chip provided in an embodiment of the present application. Figure 1As shown in , the AI ​​chip includes multiple data storage spaces and computing units. Among them, the data storage space includes global memory (Blobal Memory), which usually has 1GB of data storage space; unified buffer (Unified Buffer, UB), which usually has 256KB of data storage space; L1 buffer, which usually has 1024KB of data storage space; L0A storage unit, L0B storage unit, which usually has 64KB of data storage space; L0C storage unit, which usually has 256KB of data storage space and storage conversion unit. The computing unit includes a matrix calculation unit (CUBE Unit), an accumulator (Accelerator), a vector calculation unit (Vector Unit) and a scalar calculation unit (Scalr Unit). In a common calculation process, two input matrices to be multiplied are input from the global memory, and pass through the unified buffer, storage conversion unit, L1 buffer, storage conversion unit in turn, and then input into the L0A storage unit and L0B storage unit respectively, and then the two input matrices are multiplied by the matrix calculation unit, and then the matrix multiplication result is input into the L0C storage unit.

[0075] Among them, the above-mentioned AI chips only support half-precision matrix multiplication operations. For example, the 910A model of the Ascend AI processor is a typical chip that only supports half-precision matrix multiplication operations. When this type of chip uses FP16 to simulate FP32 matrix multiplication operations, the operation process is the same as many chips that specialize in accelerating matrix multiplication. The process is as follows:

[0076] 1. Data transfer: transfer FP32 matrices A and B from main memory GM to UB;

[0077] 2. Data calculation: Convert the FP32-format matrix A and matrix B to the FP16 format on the VEC module;

[0078] 3. Data transfer: transfer the FP16 matrix A and matrix B to L0A and L0B;

[0079] 4. Data calculation: Perform matrix multiplication and addition operations in FP16 format on the CUBE module;

[0080] 5. Data transfer: transfer the calculation results to the main memory GM.

[0081] When the amount of data in matrix operations is particularly large, the size of the blocks becomes a factor that restricts performance improvement due to the limitations of the processor's on-chip cache. The efficiency of data handling also becomes a factor that restricts performance improvement because the time allocation for data handling and data calculation is unreasonable. And because the operation process of calculating matrix multiplication on the Ascend AI processor is basically fixed, the pipeline optimization of specific algorithms must be handled separately. At this time, the conventional optimization methods provided by the processor are not applicable.

[0082] In conjunction with the above-mentioned parallel computing hardware, the data processing method for matrix operations in the parallel computing hardware provided in the embodiment of the present application and related equipment will be further described below. The data processing method for matrix operations in the parallel computing hardware provided in the embodiment of the present application can be applied to the parallel computing hardware, and can also be applied to the AI ​​chip in the parallel computing hardware.

[0083] The following is a detailed description of the data processing method for matrix operations in parallel computing hardware in the embodiment of the present application. Figure 2 , which is an optional flow chart of a data processing method for matrix operations in parallel computing hardware provided in an embodiment of the present application, Figure 2 The method may include but is not limited to steps 100 to 500. It can also be understood that this embodiment Figure 2 The order of step 100 to step 500 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0084] Step 100: Obtain a first half-precision matrix of the first single-precision matrix and a second half-precision matrix of the second single-precision matrix.

[0085] The following is a detailed description of step 100.

[0086] In some embodiments, in parallel computing hardware, in response to the first single-precision matrix A single and the second single precision matrix B single When performing a multiplication operation on a single-precision matrix, the AI ​​chip in the parallel computing hardware cannot directly process the matrix multiplication operation of a single-precision matrix. Therefore, the parallel computing hardware first converts the first single-precision matrix A single and the second single precision matrix B single The unified buffer UB is input from the global buffer, and then the vector calculation unit Vec performs the first single-precision matrix A single and the second single precision matrix B single Perform half-precision matrix conversion to obtain the first single-precision matrix A single The first half-precision matrix A half and the second single precision matrix B single The second half-precision matrix B half Then use the first half-precision matrix Ahalf and the second half-precision matrix B half Perform matrix multiplication.

[0087] In order to further improve the accuracy of matrix multiplication of two single-precision matrices on parallel computing hardware, the first single-precision matrix A is also used single Convert to the first half-precision matrix A half The first error matrix ΔA generated in the process half , and using the second single precision matrix B single Convert to the second half-precision matrix B half The second error matrix ΔB generated in the process half As a compensation term to improve the accuracy of matrix multiplication.

[0088] Reference Figure 3 , is a schematic diagram of a single-precision matrix multiplication process provided by an embodiment of the present application. Figure 3 As shown in single and the second single precision matrix B single The multiplication operation is divided into three parts. Among them, the first half-precision matrix A half and the second half-precision matrix B half The multiplication operation between the first half-precision matrix A half and the second error matrix ΔB half The multiplication operation between the second half-precision matrix B half And the first error matrix ΔA half The multiplication operation between them is performed, and the first single-precision matrix A is obtained after accumulation. single and the second single precision matrix B single The result of the multiplication operation between them.

[0089] It can be understood that when the first single precision matrix A single and the second single precision matrix B single When the matrix dimension is too large, the corresponding first half-precision matrix A half , the second half-precision matrix B half , the first error matrix ΔA half and the second error matrix ΔB half Since the storage space of the L0A storage unit and the L0B storage unit in the AI ​​chip is limited and it is impossible to store the complete data of a single matrix at one time, it is necessary to divide the matrix into multiple block matrices (such as dividing the first half-precision matrix A half Divide into multiple first block matrices And pass through the unified buffer, storage conversion unit, L1 buffer, storage conversion unit in turn, and then enter the L0A storage unit and L0B storage unit respectively, and finally enter the matrix calculation unit for matrix calculation to obtain the calculation result of each block, and finally accumulate all the block calculation results to obtain the first single-precision matrix A single and the second single precision matrix B single The multiplication result C single .

[0090] Step 200: Obtain a second error matrix between a second single-precision matrix and a second half-precision matrix, obtain a plurality of second block error matrices in the second error matrix, and store them in an L1 buffer of the parallel computing hardware.

[0091] The following is a detailed description of step 200.

[0092] In some embodiments, by Figure 3 From the analysis of the matrix multiplication process shown in the figure, it can be seen that data handling and data calculation are interspersed. If the input matrix (such as the first single-precision matrix A single and the second single precision matrix B single ) is particularly large, it needs to be divided into blocks. If you want to improve performance, you need to plan the data in advance and distribute it on multiple cores. Data segmentation and the use of multi-core computing will increase the complexity of data handling and data computing, causing data handling and data computing to be coupled with each other. At this time, if data handling is relatively slow relative to computing (that is, there is a memory access wall problem in memory access), the processor of the AI ​​chip will often need to wait to read data from the memory or write data to the memory, resulting in an increase in the idle waiting time of the processor. In addition, since two levels of cache are provided on AI chips (such as Ascend processors): L0 buffer and L1 buffer, the bandwidth of the L0 buffer to buffer data to the global buffer is much faster than the rate between the L1 buffer and the global cache, resulting in the bottleneck of data handling appearing in the L1 buffer.

[0093] It is understandable that the memory wall problem is aimed at matrix multiplication operations. It means that in a computer system, the performance improvement brought about by the CPU processing speed is much faster than the performance improvement brought about by the memory access speed, resulting in an increase in the processor's waiting time. The improvement in the performance of matrix multiplication operations is mainly restricted by the memory access speed.

[0094] In addition, the cache capacity of the L1 buffer on the AI ​​chip is 1MB, which is much larger than the cache capacity of the unified buffer UB, which is 256KB. In addition, the matrix block and vector calculation unit VEC need to frequently use the unified buffer UB. Therefore, this embodiment uses the L1 buffer as an intermediate cache in the matrix operation process. This will be further described below.

[0095] Because Figure 3 The operation process of the three multiplication operations shown in the AI ​​chip is similar. Therefore, in this embodiment, the first half-precision matrix A half and the second error matrix ΔB half The multiplication operation between them is described in detail, and the remaining two parts can be obtained similarly.

[0096] In some embodiments, the second single-precision matrix B is obtained by using the vector computing unit in the uniform buffer. single Convert to multiply the second half-precision matrix B half Then, using the second single-precision matrix B single Subtract the second half-precision matrix B half The second single-precision error matrix ΔB obtained after single-precision conversion single =B single -to-single(B half ), and then convert the second single-precision error matrix to half-precision to obtain the second error matrix ΔB half , assuming that the second single-precision matrix B single If it is too large, the second error matrix ΔB half Including multiple second block error matrices Wherein K is the number of second block error matrices stored in the L1 buffer at one time.

[0097] Then, the first half-precision matrix A half and the second error matrix ΔB half Before the multiplication operation between the two, the second block error matrix Pre-stored in the L1 buffer of the parallel computing hardware, so that in the actual first half-precision matrix A half and the second error matrix ΔB half In the multiplication operation between them, the data storage and transportation process of the L1 buffer is reduced to overcome the memory access wall problem mentioned above, and the burden of the unified buffer UB is reduced, thereby effectively improving the calculation rate of matrix operation data processing in the actual process.

[0098] The step of obtaining a plurality of second block error matrices in the second error matrix and storing them in the L1 buffer of the parallel computing hardware includes the following steps 210 to 220 .

[0099] Step 210: Determine a first storage quantity based on a first block size of the first block matrix and a second block size of the second block error matrix.

[0100] Step 210 is described in detail below.

[0101] In some embodiments, the L1 buffer is used as an intermediate cache during matrix operations. half and the second error matrix ΔB half Before the multiplication operation between the two, it is used to pre-store multiple second block error matrices In order to further improve the utilization rate of the storage space of the L1 buffer, it is necessary to appropriately divide the L1 buffer into multiple storage spaces in advance, including a storage space for storing the second block error matrix and the number of spaces used to store the first block matrix.

[0102] Therefore, in this embodiment, based on the first block matrix The first block size b m ×b k and the second block error matrix The second block size b k ×b n , to determine the first storage quantity N in the L1 buffer for storing the second block error matrix quantity L1 .

[0103] The process of determining the first storage quantity based on the first block size of the first block matrix and the second block size of the second block error matrix includes the following steps 211 to 212.

[0104] Step 211: Obtain the error matrix storage capacity based on the difference between the cache capacity of the L1 buffer and twice the size of the first block.

[0105] Step 212: Obtain a first storage quantity based on the ratio of the error matrix storage capacity to the second block size.

[0106] Steps 211 to 212 are described in detail below.

[0107] In some embodiments, when performing the first half-precision matrix A half and the second error matrix ΔB half Before the multiplication operation between the two, first based on the cache capacity of the L1 buffer (ie 1024KB) and twice the first block size 2b m b k The difference is obtained, and the error matrix storage capacity is 1024-2b m b k Next, based on the error matrix storage capacity and the second block size b k b n The ratio of the first storage quantity N is obtained. L1 =(1024-2b m b k ) / (b kb n ).

[0108] From the first storage quantity N L1 As can be seen from the formula, the L1 buffer is divided into N L1 +2 storage spaces, including N L1 A storage space is used to pre-store the second block error matrix The other two storage spaces are used to actually perform the first half-precision matrix A half and the second error matrix ΔB half When performing multiplication operations between

[0109] Step 220: Store the second block error matrix corresponding to the first storage quantity into the L1 buffer.

[0110] Steps 210 to 220 are described in detail below.

[0111] In some embodiments, after the appropriate storage space is divided in the L1 buffer, the first storage quantity (ie, N L1 ) is stored in the L1 buffer for intermediate caching.

[0112] In addition, for the second error matrix ΔB half A column in will be divided into K / N L1 groups, the second block error matrix in each group The number of L1 . All second block error matrices in each group The data will be cleaned uniformly and moved into the L1 buffer at one time to complete the optimization task of reusing the L1 buffer as an intermediate cache.

[0113] It is understandable that the second error matrix ΔB is first traversed half , because in the first half-precision matrix A half and the second error matrix ΔB half In the multiplication operation between, the second error matrix ΔB is obtained half The computational complexity is higher than that of obtaining the first half-precision matrix A half Therefore, in this embodiment, in the first half-precision matrix A half and the second error matrix ΔB half In the multiplication operation between, the second error matrix ΔB is obtained. half Only the vector calculation unit VEC is involved in the calculation of this part, which is taken out separately to prevent coupling with the calculation of the matrix calculation unit CUBE, thereby effectively accelerating the first half-precision matrix A halfand the second error matrix ΔB half The computational speed at which matrix multiplication operations are performed.

[0114] In addition, by using the L1 buffer as an intermediate cache reused in the multiplication operation of the block matrix, the pressure on the use of the unified buffer UB can be effectively reduced, and by using the larger storage space of the L1 buffer, larger matrix blocks can be obtained when performing matrix division, and larger blocks will reduce the number of data transfer times and the number of times the matrix calculation unit CUBE is called, which is conducive to accelerating the rate of matrix multiplication operations. In addition, by using the L1 buffer as an intermediate cache reused in the multiplication operation of the block matrix, more data can be stored at one time, that is, most of the storage space is used to pre-store the second error matrix, so that the number of cross-executions of data transfer and data calculation is reduced, which is more conducive to the performance of matrix multiplication, and then effectively improves the calculation speed of the matrix multiplication operation of the first half-precision matrix and the second error matrix, so as to improve the calculation rate of matrix operations in parallel computing hardware.

[0115] Step 300: Obtain the first block matrices in the first half-precision matrix one by one, store them in the L1 buffer, transfer the first block matrix and the second block error matrix in the L1 buffer to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first block result matrix, and obtain a first result matrix of the first half-precision matrix and the second error matrix based on the first block result matrix.

[0116] The following is a detailed description of step 300.

[0117] In some embodiments, a plurality of second block error matrices are pre-set After being transferred to the L1 buffer for storage, the actual second block error matrix needs to be With the first block matrix Matrix operations.

[0118] Next, get the first half-precision matrix A one by one half The first block matrix in Stored in the L1 buffer, the first block matrix in the L1 buffer and the second block error matrix The matrix calculation module transmitted to the parallel computing hardware performs matrix multiplication to obtain a first block result matrix.

[0119] Normally, when a single buffer is set in the L1 buffer for buffering the block matrix, the data handling and data calculation of the block matrix are strictly serial. To calculate the first half-precision matrix and the second error matrix For example, we need to first transform the first block matrix and the second block error matrix Move them one by one to the unified buffer UB, and then clean the data through the vector computing unit VEC calculation unit (to obtain FP16 format and ), and then further calls the matrix calculation unit CUBE to perform matrix multiplication and addition operations to obtain block calculation results, and then writes the block calculation results back to the global cache. Among them, each step of calculation must wait until the data transfer of the previous step is completed, which seriously affects the calculation efficiency.

[0120] Therefore, in the embodiment of the present application, according to the first storage quantity N L1 As can be seen from the formula, the L1 buffer gives the first block matrix Two storage spaces are divided, including a first storage location and a second storage location, for the first block matrix The cache is double-buffered, thereby separating part of the data handling and data calculation, and realizing parallel processing of data handling and data calculation to accelerate the calculation of matrix multiplication.

[0121] Among them, the first block matrix in the first half-precision matrix is ​​obtained one by one and stored in the L1 buffer, and the first block matrix and the second block error matrix in the L1 buffer are transmitted to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain the first block result matrix, including the following steps 310 to 320.

[0122] Step 310: According to the arrangement order of the plurality of first block matrices in the first half-precision matrix, obtain a second first block matrix and store it in a first storage location.

[0123] Step 320: The first first block matrix at the second storage location and the second block error matrix corresponding to the first first block matrix in the L1 buffer are transmitted to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first block result matrix.

[0124] Steps 310 to 320 are described in detail below.

[0125] In some embodiments, first, according to the first half-precision matrix A half Multiple first block matrices in The order of arrangement is to transform each first block matrix one by one Stored to the first storage location or the second storage location in the L1 buffer, and in the next first block matrix When storing to a certain storage location in the L1 buffer, the first block matrix of the previous storage in another storage location The second block error matrix corresponding to The process of transferring the matrix to the matrix calculation unit CUBE for matrix multiplication. That is, the second first block matrix When storing to the first storage location (or second storage location) of the L1 buffer, in the parallel operation hardware, the first first block matrix of the second storage location (or first storage location) in the L1 buffer will be synchronously executed and the first block matrix in the L1 buffer The corresponding second block error matrix The matrix calculation module CUBE is transmitted to the parallel computing hardware for matrix multiplication to obtain the first first block matrix The corresponding second block error matrix The first block result matrix is ​​obtained by multiplying .

[0126] Reference Figure 4 , is a schematic diagram of a double buffer optimization algorithm provided by an embodiment of the present application. Figure 4 As shown in , it lists the relevant implementation codes of steps 310 to 320, wherein the block matrix of the next first single-precision matrix is Move the block matrix from the global cache GM to the uniform buffer UB Convert to the corresponding first block matrix And further the first block matrix The first block matrix is ​​transferred to the first storage location in the L1 buffer through the unified buffer UB, and then further transferred to the L0A storage unit, and the first block matrix is ​​transferred to the first storage location in the L1 buffer. The corresponding second block error matrix In the matrix calculation module CUBE, matrix multiplication is performed to obtain the first block result matrix. L1 indivual N L1 indivual The blocks are divided into 2 groups, numbered incrementally starting from 0, and the even-numbered blocks use the first storage location reserved in the L1 buffer (marked as L1 A0 ), odd numbers use the second storage location reserved in the L1 buffer (labeled L1 A1 ). In this way, when L1 A0 Provides the current matrix multiplication calculation required When L1 A1 You can read the next This enables data transfer and data calculation to be performed simultaneously at the same time. Figure 4 The algorithm line in the black box in the figure) can be implemented at the same time through the double cache space setting in the L1 buffer, thereby effectively improving the computational efficiency of matrix multiplication in the parallel computing unit.

[0127] In the conventional FP16 simulation FP32 matrix multiplication algorithm, for a block calculation result, A half ×ΔB half and ΔA half ×B half are mixed and involved in the calculation. Figure 3 In the schematic analysis of the process of single-precision matrix multiplication shown in the figure, due to the first error matrix ΔA half and the second error matrix ΔB half The value is too small, and it is necessary to and the second block error matrix Before performing the multiplication operation, the magnification factor SF is first multiplied to reduce the error. However, this process requires many multiplication operations of the magnification factor SF, thereby reducing the computational efficiency of matrix multiplication in parallel computing hardware. In this embodiment, in order to solve this problem, multiple storage spaces pre-divided by the L1 buffer are used to improve it, as described below.

[0128] The first block matrix and the second block error matrix in the L1 buffer are transmitted to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain the first block result matrix, and the following steps 330 to 340 are also included.

[0129] Step 330: In the L1 buffer, a plurality of second block error matrices are multiplied by an amplification factor to obtain an amplified second block error matrix.

[0130] Step 340: The enlarged second block error matrix corresponding to the first block matrix in the L1 buffer is transmitted to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first block result matrix.

[0131] Steps 330 to 340 are described in detail below.

[0132] In some embodiments, for A half ×ΔB half In the operation, the L1 buffer is pre-reasonably divided into multiple storage spaces and multiple second block error matrices are pre-cached After that, multiple second block error matrices in the L1 buffer are At the same time, the magnification factor SF is multiplied to convert the error matrices of multiple second blocks in the L1 buffer into All become the corresponding enlarged second block error matrix This effectively reduces the actual A half ×ΔB half An operation that multiplies the amplification factor multiple times during calculation.

[0133] After obtaining multiple amplified second block error matrices After that, the first block matrix in the L1 buffer is The corresponding enlarged second block error matrix The matrix calculation module transmitted to the parallel computing hardware performs matrix multiplication to obtain the corresponding first block result matrix.

[0134] Reference Figure 5 , is a flowchart of a block operation of a single-precision matrix multiplication provided in an embodiment of the present application. Figure 5 As shown in FIG. 1 , two second block error matrices are pre-stored in the L1 buffer. Or two first block error matrices For example, in the operation flow, the two stored second block error matrices are preliminarily (or two first block error matrices ) is multiplied by the magnification factor SF, and then the multiplication operation is performed. Figure 5 It can be seen from the multiplication operation flow shown that the magnification factor SF only participates in 5 calculations (1 calculation each time the L1 buffer cache is reused, 4 times the L1 buffer is reused for a total of 4 times and finally participates in 1 division operation). Figure 3 The multiplication process shown in the figure reduces the calculation of the magnification factor SF by 4 times. Figure 3 In the process shown in FIG. 1 , the number of times the magnification factor SF participates in the calculation is equal to 2×N+1. After optimization (i.e., Figure 5 In the process shown in the figure), the number of times the magnification factor SF participates in the calculation is equal to 2×(N / N L1 )+1. After optimization, it can reduce 2×N+1-(2×(N / N L1 )+1)=2×N(1-1 / N L1 ) times of calculation. It can be understood that, usually, the first storage quantity N pre-divided in the L1 buffer L1 are large, so by Figure 5 The process shown will significantly reduce the number of times the magnification factor SF participates in the calculation, thereby effectively improving the operating efficiency of the matrix multiplication while ensuring the accuracy of the matrix multiplication in the parallel computing hardware.

[0135] Reference Figure 6 , is a schematic diagram of an algorithm flow of a multiplication operation of a half-precision matrix and an error matrix provided in an embodiment of the present application. Figure 6 As shown, this is the multiplication operation A of the first single precision matrix and the second error matrix half ×ΔB halfThe algorithm flow chart specifically mentions the relevant flow description of the above-mentioned steps 100 to 300, that is, it mentions that the L1 buffer is used as a multiplexing intermediate buffer and a first block matrix is ​​set in the L1 buffer. The double cache location effectively ensures the accuracy of matrix multiplication in parallel computing hardware while improving the operating efficiency of matrix multiplication.

[0136] In addition, since instructions from different functional units on AI chips (such as Ascend processors) are usually assigned to different pipelines for execution, the timing relationship between these pipelines cannot be automatically guaranteed to be correct. Even for the same pipeline, Ascend processors cannot guarantee the dependencies between them. The following lists the pipelines used in matrix multiplication.

[0137] Reference Figure 7 , is a schematic diagram of a pipeline configuration provided by an embodiment of the present application. Figure 7 As shown in , the default name of the pipeline for performing vector operations on source data is PIPE_V, the default name of the pipeline for performing matrix operations on source data is PIPE_M, the default name of the pipeline for transferring source data from the unified buffer UB to the global buffer GM or the L1 buffer is PIPE_MTE3, the default name of the pipeline for transferring source data from the unified buffer UB to the unified buffer UB or the L0C storage unit is PIPE_V, and so on.

[0138] In common AI chips (such as Ascend hardware platforms), manual pipeline is disabled by default, that is, automatic pipeline is used, and the execution dependency of each computing unit is handled by hardware. Figure 6 In the algorithm flow shown, although the automatic pipeline can ensure the correctness of the timing relationship, it cannot achieve the best result, which will seriously affect the overall performance of matrix multiplication.

[0139] Therefore, in this embodiment, the dependencies between data handling and data calculation pipelines are manually specified to simplify the pipeline dependencies, thereby ensuring the correctness of the timing relationship and improving the performance of matrix multiplication in parallel computing hardware.

[0140] Wherein, before step 100: obtaining the first half-precision matrix of the first single-precision matrix and the second half-precision matrix of the second single-precision matrix, the data processing method for matrix operations in parallel computing hardware further includes the step of: pipeline locking the block error module.

[0141] After step 200: obtaining multiple second block error matrices in the second error matrix and storing them in the L1 buffer of the parallel computing hardware, the data processing method for matrix operations in the parallel computing hardware further includes the step of pipeline unlocking the block error module.

[0142] In step 300 , before obtaining the first block matrices in the first half-precision matrix one by one and storing them in the L1 buffer, the data processing method for matrix operations in parallel computing hardware further includes the step of: pipeline locking the block computing module.

[0143] And, after step 300: obtaining the first result matrix of the first half-precision matrix and the second error matrix based on the first block result matrix, the data processing method of matrix operations in parallel computing hardware also includes the step of: pipeline unlocking the block computing module.

[0144] In some embodiments, the block error module includes a second block error matrix generation and storage to the L1 buffer process, that is, in the case of Figure 6 In the algorithm flow shown, the second block error matrix is ​​obtained The corresponding for loop in the process of storing the data in the L1 buffer is used as an MTE3 pipeline (i.e., a block error module). It can be understood that the process can be abstracted as an MTE3 pipeline because the purpose of this for loop is to obtain the second block error matrix and stored in the L1 buffer, which is exactly a PIPE_MTE3 pipeline; in addition, Figure 6 In the algorithm flow shown, the subsequent use of the L1 buffer does not involve the relevant operations of the PIPE_MTE3 pipeline. This example is just a simple example. When abstracting multiple data transfers and calculations, different abstraction methods will have different effects on performance.

[0145] In addition, the block calculation module includes a first block matrix stored in the L1 buffer, and a matrix calculation process of the first block matrix and the second block error matrix, that is, in the following example: Figure 6 In the algorithm flow shown, the first block matrix Generation, storage to the L1 buffer and the corresponding second block error matrix The process of performing multiplication operations is abstracted as the MTE1 pipeline (i.e., block calculation module).

[0146] In this embodiment, the block error module and the block calculation module are isolated from each other, so as to avoid the second block error matrix in a pipeline from the perspective of timing flow. The change causes another pipeline to operations are affected, thereby effectively improving the correctness of ensuring the timing relationship and improving the accuracy of matrix multiplication in parallel computing hardware.

[0147] The present application embodiment is through Figure 6 In the algorithm flow shown, multiple for loops are abstracted into two pipeline modules (including a block error module and a block calculation module). This abstraction has two advantages: on the one hand, the dependencies between pipelines involved in the for loop are simplified. On the other hand, the interior of a for loop is abstracted into a pipeline, and the for loop does not involve external pipelines, and the pipeline dependencies within the for loop are also simplified. By abstracting the for loop into a pipeline, the pipeline relationships inside and outside the for loop are simplified, which speeds up the operation of matrix multiplication.

[0148] Reference Figure 8 , is another algorithm flow diagram of the multiplication operation of a half-precision matrix and an error matrix provided in an embodiment of the present application. Figure 8 As shown, it is based on Figure 6 The algorithm flow shown is combined with the algorithm flow generated by manually specifying the dependencies between data handling and data calculation pipelines, that is, Figure 8 The first dotted box in the figure is used as a block error module. During the process of executing the block error module, the critical section between the pipeline MTE3 corresponding to the block error module and the pipeline MTE1 corresponding to the block calculation module is locked until all function operations corresponding to the block error module are executed; then the critical section between the pipeline MTE3 and the pipeline MTE1 is unlocked. Figure 8 The second dotted box shown in the figure is used as a block calculation module. When executing the block calculation module, similar to executing the block error module, the critical section between pipeline MTE3 and pipeline MTE1 is also locked until all function operations in the block calculation module are executed.

[0149] Reference Fig. 9 , is a schematic diagram of a dual cache memory switching process provided by an embodiment of the present application. Based on the switching of critical sections corresponding to multiple modules in the algorithm flow generated by combining the dependencies of manually specified data handling and data calculation pipelines, the advantages of the dual cache mechanism set in the L1 buffer are further described. Fig. 9 As shown in A half ×ΔB halfTaking the example of dividing into 4 blocks, double buffering involves 2 blocks of memory (2 blocks is a general term. In fact, all variables involved in the calculation must be allocated 2 non-overlapping memory blocks). At the same time, one is responsible for calculation and the other is responsible for data transfer. Therefore, special treatment is required when specifying manual pipelines. This application uses the following scheme to optimize the double buffering pipeline: (1) All pipelines involved in the double buffering are abstracted into an M pipeline (i.e., Figure 8 (2) Apply for two critical sections. The two blocks of memory used by the double cache are protected by different critical sections. The two critical sections alternately protect the pipelines involved in the double cache, thereby effectively ensuring the accuracy of matrix multiplication in parallel computing hardware while improving the operating efficiency of matrix multiplication.

[0150] The step of accumulating the first block result matrix corresponding to each first block matrix to obtain the first result matrix of the first half-precision matrix and the second error matrix includes the following steps 350.

[0151] Step 350: Accumulate the first block result matrix corresponding to each first block matrix to obtain a first result matrix of the first half-precision matrix and the second error matrix.

[0152] under Step 350 is described in detail.

[0153] In some embodiments, in A half ×ΔB half In the operation process of The corresponding second error block matrix After performing matrix operations to obtain the first block result matrix, accumulating all first block result matrices can obtain the first half-precision matrix A half and the second error matrix ΔB half The first result matrix obtained by performing the matrix operation.

[0154] The above is the multiplication operation A of the first half-precision matrix and the second error matrix half ×ΔB half The following part is an example, and the rest is the multiplication operation of the first half-precision matrix and the second half-precision matrix A. half ×B half part, and the multiplication operation ΔA of the first error matrix and the second half-precision matrix half ×B half Some of the processing procedures are similar.

[0155] Step 400: Obtain a first error matrix between a first single-precision matrix and a first half-precision matrix.

[0156] The following is a detailed description of step 400.

[0157] In some embodiments, after performing the multiplication operation A of the first half-precision matrix and the second error matrix half ×ΔB half In addition to the first error matrix, the multiplication operation ΔA of the second half-precision matrix needs to be performed. half ×B half Part of the processing flow is the same as the above to obtain the second error matrix ΔB half Similarly, the first single-precision matrix A is obtained using the vector calculation unit in the uniform buffer single Convert to multiply the first half-precision matrix A half Then, using the first single precision matrix A single Subtract the first half-precision matrix A half The first single-precision error matrix ΔA obtained after single-precision conversion single =A single -to-single(A half ), and then convert the first single-precision error matrix to half-precision to obtain the first error matrix ΔA half , assuming that the first single-precision matrix B single If it is too large, the first error matrix ΔA half Including multiple first block error matrices

[0158] Step 500: Obtain a second result matrix obtained by matrix multiplication of the first error matrix and the second half-precision matrix, obtain a third result matrix obtained by matrix multiplication of the first half-precision matrix and the second half-precision matrix, accumulate the first result matrix, the second result matrix and the third result matrix to obtain a single-precision target matrix, and use the single-precision target matrix as the result of matrix multiplication of the first single-precision matrix and the second single-precision matrix.

[0159] The following is a detailed description of step 500.

[0160] In some embodiments, after performing the multiplication operation A of the first half-precision matrix and the second error matrix half ×ΔB half In addition to the first error matrix, after obtaining the first error matrix, it is also necessary to obtain the first error matrix and the second half-precision matrix similar to the above steps 200 to 300 to perform matrix multiplication ΔA half ×B half The second result matrix is ​​obtained, and the first half-precision matrix and the second half-precision matrix are obtained to perform matrix multiplication A half ×B halfFinally, the first result matrix, the second result matrix and the third result matrix are accumulated to obtain the half-precision target matrix C half , and then convert the half-precision target matrix into a single-precision matrix C single , and uses the single-precision destination matrix as the result of the matrix multiplication of the first single-precision matrix and the second single-precision matrix.

[0161] Among them, A is performed successively for the same AI chip. half ×ΔB half Partial and ΔA half ×B half When performing some operations, since A is performed first half ×ΔB half During some operations, the amount of space planned in the L1 buffer in the AI ​​chip is based on the second block half-precision matrix The matrix size b k b n The goal is to accommodate more second block half-precision matrices. However, after ΔA half ×B half During partial operation, the first block error matrix The matrix size b m b k does not match, so it cannot be used directly and needs to be reconfigured. However, reconfiguration will further increase the computing time cost. Therefore, in this embodiment, for the subsequent ΔA half ×B half During partial operations, the first block of half-precision matrices is computed based on the size of the pre-divided L1 buffer space. The matrix size and the second block matrix The matrix size is adaptively adjusted to improve the calculation rate. The details are as follows.

[0162] The process of obtaining a first result matrix of a first half-precision matrix and a second error matrix based on the first block result matrix includes the following steps 510 to 540.

[0163] Step 510: Obtain a third block size according to the first block size, and obtain a fourth block size according to the second block size.

[0164] Step 520: Split the second half-precision matrix according to the third block size to obtain a plurality of second block matrices.

[0165] Step 530: split the first error matrix according to the fourth block size to obtain multiple first block error matrices.

[0166] Step 540: Obtain a second block result matrix obtained by performing matrix multiplication of each second block matrix and the first block error matrix through the L1 buffer and the matrix calculation module one by one, and obtain a second result matrix based on all the second block result matrices.

[0167] Steps 510 to 540 are described in detail below.

[0168] In some embodiments, after executing A half ×ΔB half After some operations, ΔA half ×B half In order to directly convert multiple first block error matrices into Put it directly into the L1 buffer and convert it into the original first block matrix The first block size b m b k Get the third block size b k b m , and according to the original second error block matrix The second block size b k b n Get the fourth block size b n b k .

[0169] Next, according to the third block size b k b m For the second half-precision matrix B half Split to obtain multiple second block matrices And according to the fourth block size b k b n For the first error matrix ΔA half Split to obtain multiple first block error matrices It can be understood that the second block matrix at this time The size is the third block size b k b m , the first block error matrix The size is the fourth block size b k b n The first storage quantity N at this time L1 The formula will become N L1 =(1024-2b k b m ) / (b n b k ). Since the block size b m 、b k 、b nIt can be adjusted dynamically, reusing the same set of algorithm processing logic and code, so that optimization can take effect on each module. The layout of the L1 buffer does not need to be changed, and it can be applied to more scenarios. At the same time, the memory layout of the UB does not need to be changed, which can effectively improve the computing rate in parallel computing hardware.

[0170] Then, similar to the related steps of steps 200 to 300 above, each second block matrix and the first block error matrix are obtained one by one through matrix multiplication through the L1 buffer and the matrix calculation module to obtain the second block result matrix, and the second result matrix is ​​obtained based on all the second block result matrices.

[0171] In order to further verify the reliability of the data processing method for matrix operations in parallel computing hardware provided by the present application, the embodiment of the present application also performs simulation performance verification on the data processing method for matrix operations in parallel computing hardware.

[0172] In some embodiments, the block size and data handling efficiency can affect the performance of matrix multiplication calculations. The capacity limits of the unified buffer UB cache (256KB) and the L1 buffer (1024KB) make it impossible for the block size and the number of data stored in the L1 buffer to be infinite. According to the hardware requirements of the Ascend processor: 1. The CUBE computing unit requires that the block size must be a multiple of 16, and 2. The capacity limit of L0C is 256KB. Under the premise of the above hardware constraints, in order to verify the improvement of matrix multiplication performance in this application and verify how high the performance can be achieved, large blocks are selected as much as possible, and the efficiency of multiplexing the L1 buffer (that is, the number of FP16 data that the L1 buffer can store) is maximized as much as possible. The following data points are selected for testing. During the test, all 32 AI chips in the parallel running hardware are used, and the input size of the test data is: M = number of AI chips × b m ×NUM, N = number of AI chips × b n ×NUM,K=b k ×2×N L1 ×NUM, where: NUM is to use as large an input as possible for testing. According to the limitation of global buffer GM=1GB, NUM=6 for this test.

[0173] Reference Fig.10 , is a parameter diagram of a performance simulation of a data processing method provided in an embodiment of the present application. Fig.10 As shown in m 、b k and b n is the size of the matrix block, N L1 It is the maximum number of blocks that can be placed in the L1 cache at one time, and the L1 usage capacity is the number of FP16 data stored in the L1 cache.

[0174] In order to verify the effect, the evaluation formula used is: relative residual = (||C FP64 -C NPU || F ) / ||C FP64 || F .

[0175] where ||·|| F represents the L2 norm, C FP64 is the result of the OpenBLAS library's cblas_dgemm calculation, C NPU This is the calculation result on the Ascend 910A processor. The actual effect of the optimization of this patent is verified from three directions. The detailed verification results are as follows.

[0176] Reference Fig.11 , is a performance simulation diagram of the first data processing method provided in the embodiment of the present application. Fig.11 As shown in , by comparing the accuracy with the BLAS library running results, it can be found that when using block calculation matrix multiplication, the performance optimization scheme used in this application is basically consistent with the BLAS library's sgemm in terms of matrix multiplication accuracy. When the BLAS library's sgemm function is not block-based, the performance optimization scheme used in this patent is more accurate. This shows that the optimization algorithm of this application can meet the accuracy requirements of single-precision matrix multiplication.

[0177] Reference Fig.12 , is a performance simulation diagram of the second data processing method provided in the embodiment of the present application. Fig.12 As shown in , by continuously adding optimization schemes to the optimization algorithm of this application, it can be observed that the optimization scheme has an efficiency in improving the performance of single-precision matrix multiplication operations. When no optimization scheme is added, the average performance of the test is 3TFlops. After adding the reused L1 and double cache optimization schemes, the average performance of the test is 15.08TFlops, and the performance is improved by about 5 times. After further adding the manual assignment pipeline optimization scheme, the average performance of the test is 52.85TFlops, and the performance is improved by about 3.5 times again. Through the optimization scheme of this patent, the performance average is increased from 3TFlops to 52.85TFlops, and the overall performance is improved by about 17.6 times, which is a significant performance improvement.

[0178] Reference Fig.13 , is a performance simulation diagram of the third data processing method provided in the embodiment of the present application. Fig.13As shown in , on the 910A hardware that does not support single-precision matrix multiplication, FP16 is used to simulate single-precision matrix multiplication operations through the CUBE operation unit. Before the optimization scheme of this application is implemented, the optimal performance is 4.64TFlops; after the optimization scheme proposed in this application is implemented, the optimal performance is 62.36TFlops, which is nearly 12.4 times higher. At the same time, through the actual measurement effect analysis, the optimal block combination of the optimization scheme proposed in this patent is (b m =192, b k =80, b n =192,N L =32), the measured performance reached 73% of the equivalent single-precision matrix multiplication peak value (256TFlops / 3=85.33TFlops) on the Ascend 910A. This is a significant performance improvement on hardware that does not support single-precision matrix multiplication.

[0179] The data processing method and related device for matrix operations in parallel computing hardware proposed in the embodiment of the present application include: first, locking the block error module pipeline, obtaining a first half-precision matrix of the first single-precision matrix and a second half-precision matrix of the second single-precision matrix; then, obtaining a second error matrix between the second single-precision matrix and the second half-precision matrix, obtaining the error matrix storage capacity based on the difference between the cache capacity of the L1 buffer and twice the first block size, obtaining a first storage quantity based on the ratio of the error matrix storage capacity to the second block size, and converting the second error matrix corresponding to the first storage quantity into a second half-precision matrix; The block error matrix is ​​stored in the L1 buffer, the block error module is pipeline unlocked, and the second error matrix is ​​a half-precision matrix; then, the block calculation module is pipeline locked, and according to the arrangement order of multiple first block matrices in the first half-precision matrix, the second first block matrix is ​​obtained and stored in the first storage position, and the multiple second block error matrices are multiplied by the amplification factor in the L1 buffer to obtain the amplified second block error matrix, and the amplified second block error matrix corresponding to the first block matrix in the L1 buffer is transmitted to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain the first The block result matrix is ​​obtained, and the first block result matrix corresponding to each first block matrix is ​​accumulated to obtain the first result matrix of the first half-precision matrix and the second error matrix, and the block calculation module is pipeline unlocked; and the first error matrix between the first single-precision matrix and the first half-precision matrix is ​​obtained, and the first error matrix is ​​a half-precision matrix; finally, the third block size is obtained according to the first block size, and the fourth block size is obtained according to the second block size, the second half-precision matrix is ​​split according to the third block size to obtain multiple second block matrices, and the first error matrix is ​​split according to the fourth block size to obtain multiple first block error matrices, and each second block matrix and the first block error matrix are obtained one by one through the L1 buffer and the matrix calculation module to perform matrix multiplication to obtain the second block result matrix, and the second result matrix is ​​obtained based on all the second block result matrices, and the third result matrix is ​​obtained to obtain the first half-precision matrix and the second half-precision matrix. The matrix is ​​accumulated to obtain the single-precision target matrix, and the single-precision target matrix is ​​used as the result of the matrix multiplication operation between the first single-precision matrix and the second single-precision matrix.

[0180] The embodiment of the present application can effectively reduce the pressure on the use of the unified buffer UB by using the L1 buffer as an intermediate cache reused in the multiplication operation of the block matrix, and by using the larger storage space of the L1 buffer, it can divide the matrix into larger blocks when dividing the matrix, and the larger blocks will reduce the number of data transfers and the number of calls to the matrix calculation unit CUBE, which is beneficial to accelerate the rate of matrix multiplication operations. In addition, by using the L1 buffer as an intermediate cache reused in the multiplication operation of the block matrix, more data can be stored at one time, that is, most of the storage space is used to pre-store the second error matrix, so that the number of cross-executions of data transfer and data calculation is reduced, which is more beneficial to the performance of matrix multiplication, and then effectively improves the calculation speed of the matrix multiplication operation of the first half-precision matrix and the second error matrix, so as to improve the calculation rate of matrix operations in parallel computing hardware; and by setting a double cache area in the L1 buffer, data transfer and data calculation are synchronized at the "same time", that is, Figure 4 The algorithm lines in the black box in , can be implemented at the same time, so as to effectively improve the computational efficiency of matrix multiplication in the parallel computing unit; and, by Figure 5 According to the process shown in FIG. 1 , the magnification factor SF is calculated for multiple matrices in the L1 buffer together, which can significantly reduce the number of times the magnification factor SF participates in the calculation, thereby effectively improving the operation efficiency of the matrix multiplication while ensuring the accuracy of the matrix multiplication in the parallel computing hardware; in addition, by Figure 6 In the algorithm flow shown, multiple for loops are abstracted into two pipeline modules (including the block error module and the block calculation module) so that the dependencies between pipelines involved in the for loop are simplified. Since the interior of a for loop is abstracted into a pipeline, the for loop does not involve external pipelines, and the pipeline dependencies within the for loop are also simplified. By abstracting the for loop into a pipeline, the pipeline relationships inside and outside the for loop are simplified, which speeds up the operation of matrix multiplication. Finally, due to the block size b m 、b k 、b n It can be adjusted dynamically, reusing the same set of algorithm processing logic and code, so that optimization can take effect on each module. The layout of the L1 buffer does not need to be changed, and it can be applied to more scenarios. At the same time, the memory layout of the UB does not need to be changed, which effectively ensures the reliability of matrix multiplication operations while improving the calculation rate of matrix operations in parallel computing hardware.

[0181] The present application also provides a data processing device for matrix operations in parallel computing hardware, which can implement the data processing method for matrix operations in the above-mentioned parallel computing hardware, referring to Fig.14, the device 1400 comprises:

[0182] A matrix acquisition module 1410, configured to acquire a first half-precision matrix of the first single-precision matrix and a second half-precision matrix of the second single-precision matrix;

[0183] A first error matrix module 1420 is used to obtain a second error matrix between a second single-precision matrix and a second half-precision matrix, obtain a plurality of second block error matrices in the second error matrix, and store them in an L1 buffer of the parallel computing hardware, wherein the second error matrix is ​​a half-precision matrix;

[0184] The matrix calculation module 1430 is used to obtain the first block matrix in the first half-precision matrix one by one, store them in the L1 buffer, transmit the first block matrix and the second block error matrix in the L1 buffer to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first block result matrix, and obtain a first result matrix of the first half-precision matrix and the second error matrix based on the first block result matrix;

[0185] A second error matrix module 1440 is configured to obtain a first error matrix between the first single-precision matrix and the first half-precision matrix, where the first error matrix is ​​a half-precision matrix;

[0186] The result calculation module 1450 is used to obtain the second result matrix obtained by matrix multiplication of the first error matrix and the second half-precision matrix, obtain the third result matrix obtained by matrix multiplication of the first half-precision matrix and the second half-precision matrix, accumulate the first result matrix, the second result matrix and the third result matrix to obtain the single-precision target matrix, and use the single-precision target matrix as the result of matrix multiplication of the first single-precision matrix and the second single-precision matrix.

[0187] In the above embodiments, the description of each embodiment has its own emphasis. For the part that is not described in detail in a certain embodiment, the specific implementation method of the data processing device for matrix operations in parallel computing hardware is basically the same as the specific implementation method of the data processing method for matrix operations in the above parallel computing hardware, and will not be repeated here.

[0188] In the embodiment of the present application, the data processing device for matrix operations in parallel computing hardware can effectively reduce the pressure on the use of the unified buffer UB by using the L1 buffer as an intermediate cache reused in the multiplication operation of the block matrix, and by using the larger storage space of the L1 buffer, it can divide the matrix into larger blocks when dividing the matrix, and the larger blocks will reduce the number of data transfers and the number of times the matrix calculation unit CUBE is called, which is beneficial to accelerate the rate of matrix multiplication operations. In addition, by using the L1 buffer as an intermediate cache reused in the multiplication operation of the block matrix, more data can be stored at one time, that is, most of the storage space is used to pre-store the second error matrix, so that the number of cross-executions of data transfer and data calculation is reduced, which is more beneficial to the performance of matrix multiplication, and then effectively improves the calculation speed of the matrix multiplication operation of the first half-precision matrix and the second error matrix, so as to improve the calculation rate of matrix operations in parallel computing hardware; and by setting a double cache area in the L1 buffer, data transfer and data calculation are synchronized at the "same time", that is, Figure 4 The algorithm lines in the black box in , can be implemented at the same time, so as to effectively improve the computational efficiency of matrix multiplication in the parallel computing unit; and, by Figure 5 According to the process shown in FIG. 1 , the magnification factor SF is calculated for multiple matrices in the L1 buffer together, which can significantly reduce the number of times the magnification factor SF participates in the calculation, thereby effectively improving the operation efficiency of the matrix multiplication while ensuring the accuracy of the matrix multiplication in the parallel computing hardware; in addition, by Figure 6 In the algorithm flow shown, multiple for loops are abstracted into two pipeline modules (including the block error module and the block calculation module) so that the dependencies between pipelines involved in the for loop are simplified. Since the interior of a for loop is abstracted into a pipeline, the for loop does not involve external pipelines, and the pipeline dependencies within the for loop are also simplified. By abstracting the for loop into a pipeline, the pipeline relationships inside and outside the for loop are simplified, which speeds up the operation of matrix multiplication. Finally, due to the block size b m 、b k 、b n It can be adjusted dynamically, reusing the same set of algorithm processing logic and code, so that optimization can take effect on each module. The layout of the L1 buffer does not need to be changed, and it can be applied to more scenarios. At the same time, the memory layout of the UB does not need to be changed, which effectively ensures the reliability of matrix multiplication operations while improving the calculation rate of matrix operations in parallel computing hardware.

[0189] The present application also provides an electronic device, including:

[0190] at least one memory;

[0191] at least one processor;

[0192] at least one program;

[0193] The program is stored in the memory, and the processor executes the at least one program to implement the data processing method of matrix operation in the parallel computing hardware implemented in the present application. The electronic device can be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a car computer, etc.

[0194] See also Fig.15 , Fig.15 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0195] The processor 1501 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0196] The memory 1502 can be implemented in the form of ROM (Read Only Memory), static storage device, dynamic storage device or RAM (Random Access Memory). The memory 1502 can store operating systems and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1502, and the processor 1501 calls and executes the data processing method of matrix operations in the parallel computing hardware of the embodiments of this application;

[0197] Input / output interface 1503, used to implement information input and output;

[0198] Communication interface 1504, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0199] A bus 1505 that transmits information between the various components of the device (e.g., the processor 1501, the memory 1502, the input / output interface 1503, and the communication interface 1504);

[0200] The processor 1501 , the memory 1502 , the input / output interface 1503 and the communication interface 1504 are connected to each other in communication within the device via the bus 1505 .

[0201] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the data processing method for matrix operations in the above-mentioned parallel computing hardware is implemented.

[0202] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0203] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0204] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0205] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0206] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0207] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0208] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0209] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0210] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0211] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0212] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0213] The preferred embodiments of the present application are described above with reference to the accompanying drawings, but the scope of the rights of the present application is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present application should be within the scope of the rights of the present application.

Claims

1. A data processing method for matrix operations in parallel computing hardware, characterized in that: The method comprises: Obtain a first half-precision matrix of the first single-precision matrix and a second half-precision matrix of the second single-precision matrix; Obtain a second error matrix between the second single-precision matrix and the second half-precision matrix, obtain a plurality of second block error matrices in the second error matrix, and store them in an L1 buffer of the parallel computing hardware, wherein the second error matrix is ​​a half-precision matrix; Obtaining first block matrices in the first half-precision matrix one by one, storing them in the L1 buffer, transmitting the first block matrix and the second block error matrix in the L1 buffer to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first block result matrix, and obtaining a first result matrix of the first half-precision matrix and the second error matrix based on the first block result matrix; Obtaining a first error matrix between the first single-precision matrix and the first half-precision matrix, where the first error matrix is ​​a half-precision matrix; A second result matrix obtained by matrix multiplication of the first error matrix and the second half-precision matrix is ​​obtained, a third result matrix obtained by matrix multiplication of the first half-precision matrix and the second half-precision matrix is ​​obtained, the first result matrix, the second result matrix and the third result matrix are accumulated to obtain a single-precision target matrix, and the single-precision target matrix is ​​used as a result of matrix multiplication of the first single-precision matrix and the second single-precision matrix.

2. The data processing method for matrix operations in parallel computing hardware according to claim 1, characterized in that: The acquiring of a plurality of second block error matrices in the second error matrix and storing the plurality of second block error matrices in the L1 buffer of the parallel computing hardware comprises: Determining a first storage quantity based on a first block size of the first block matrix and a second block size of the second block error matrix; The second block error matrices corresponding to the first storage quantity are stored in the L1 buffer.

3. The data processing method for matrix operations in parallel computing hardware according to claim 2, characterized in that: The determining the first storage quantity based on the first block size of the first block matrix and the second block size of the second block error matrix includes: Obtaining an error matrix storage capacity based on a difference between a cache capacity of the L1 buffer and twice the size of the first block; The first storage quantity is obtained based on the ratio of the error matrix storage capacity to the second block size.

4. The data processing method for matrix operations in parallel computing hardware according to claim 3, characterized in that: The L1 buffer is provided with a first storage location and a second storage location for storing the first block matrix, the first block matrix in the first half-precision matrix is ​​obtained one by one, and stored in the L1 buffer, and the first block matrix and the second block error matrix in the L1 buffer are transmitted to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first block result matrix, including: According to the arrangement order of the plurality of the first block matrices in the first half-precision matrix, obtaining a second first block matrix and storing it in the first storage location; The first first block matrix at the second storage location and the second block error matrix corresponding to the first first block matrix in the L1 buffer are transmitted to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first block result matrix.

5. The data processing method for matrix operations in parallel computing hardware according to claim 1, characterized in that: The step of transmitting the first block matrix and the second block error matrix in the L1 buffer to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first block result matrix includes: In the L1 buffer, a plurality of the second block error matrices are multiplied by an amplification factor to obtain an amplified second block error matrix; The enlarged second block error matrix corresponding to the first block matrix in the L1 buffer is transmitted to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain the first block result matrix.

6. The data processing method for matrix operations in parallel computing hardware according to claim 2, characterized in that: Before obtaining the first half-precision matrix of the first single-precision matrix and the second half-precision matrix of the second single-precision matrix, the method further includes: Pipeline locking a block error module, wherein the block error module includes a process of generating and storing the second block error matrix in the L1 buffer; After acquiring the plurality of second block error matrices in the second error matrix and storing them in the L1 buffer of the parallel computing hardware, the method further includes: The block error module is pipeline unlocked.

7. The data processing method for matrix operations in parallel computing hardware according to claim 2, characterized in that: Before acquiring the first block matrices in the first half-precision matrix one by one and storing them in the L1 buffer, the method further includes: Pipeline locking a block calculation module, wherein the block calculation module includes storing the first block matrix into the L1 buffer, and a matrix calculation process of the first block matrix and the second block error matrix; After obtaining a first result matrix of the first half-precision matrix and the second error matrix based on the first block result matrix, the method further includes: The block computing module is pipeline unlocked.

8. The data processing method for matrix operations in parallel computing hardware according to claim 4, characterized in that: The step of obtaining a first result matrix of the first half-precision matrix and the second error matrix based on the first block result matrix includes: The first block result matrices corresponding to each of the first block matrices are accumulated to obtain a first result matrix of the first half-precision matrix and the second error matrix.

9. The data processing method for matrix operations in parallel computing hardware according to claim 2, characterized in that: The obtaining a second result matrix obtained by performing matrix multiplication of the first error matrix and the second half-precision matrix includes: Obtaining a third block size according to the first block size, and obtaining a fourth block size according to the second block size; Splitting the second half-precision matrix according to the third block size to obtain a plurality of second block matrices; Splitting the first error matrix according to the fourth block size to obtain a plurality of first block error matrices; A second block result matrix is ​​obtained by performing matrix multiplication of each second block matrix and the first block error matrix through the L1 buffer and the matrix calculation module one by one, and the second result matrix is ​​obtained based on all the second block result matrices.

10. A data processing device for matrix operations in parallel computing hardware, characterized in that: The device comprises: A matrix acquisition module, used for acquiring a first half-precision matrix of the first single-precision matrix and a second half-precision matrix of the second single-precision matrix; a first error matrix module, configured to obtain a second error matrix between the second single-precision matrix and the second half-precision matrix, obtain a plurality of second block error matrices in the second error matrix, and store them in an L1 buffer of the parallel computing hardware, wherein the second error matrix is ​​a half-precision matrix; a matrix calculation module, configured to obtain first block matrices in the first half-precision matrix one by one, store them in the L1 buffer, transmit the first block matrix and the second block error matrix in the L1 buffer to the matrix calculation module of the parallel computing hardware for matrix multiplication to obtain a first block result matrix, and obtain a first result matrix of the first half-precision matrix and the second error matrix based on the first block result matrix; A second error matrix module, obtaining a first error matrix between the first single-precision matrix and the first half-precision matrix, where the first error matrix is ​​a half-precision matrix; A result calculation module is used to obtain a second result matrix obtained by matrix multiplication of the first error matrix and the second half-precision matrix, obtain a third result matrix obtained by matrix multiplication of the first half-precision matrix and the second half-precision matrix, accumulate the first result matrix, the second result matrix and the third result matrix to obtain a single-precision target matrix, and use the single-precision target matrix as the result of matrix multiplication of the first single-precision matrix and the second single-precision matrix.

11. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the data processing method for matrix operations in the parallel computing hardware according to any one of claims 1 to 9 when executing the computer program.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data processing method for matrix operations in parallel computing hardware according to any one of claims 1 to 9 is implemented.

Images