Data processing method, device, computer equipment and storage medium

Abstract

Embodiments of the present application disclose a data processing method, apparatus, computer equipment, and storage medium. A neural network processor includes a global memory, a transfer buffer, and multiple computing units. A plurality of sub-matrices to be updated are obtained by performing block processing on a matrix to be updated, and the plurality of sub-matrices to be updated are evenly distributed to each computing unit. A first sub-matrix corresponding to each sub-matrix to be updated is determined in a first matrix, and the first sub-matrix is ​​moved from the global memory to a preset buffer of the computing unit corresponding to each sub-matrix to be updated. A second sub-matrix corresponding to each sub-matrix to be updated is determined in a second matrix, and the second sub-matrix is ​​moved from the global memory to the transfer buffer. A target first sub-matrix is ​​obtained from the preset buffer, and a target second sub-matrix is ​​obtained from the transfer buffer. A general matrix multiplication operation is performed on each sub-matrix to be updated, a target first sub-matrix, and a target second sub-matrix by the computing unit to obtain an updated sub-matrix.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, apparatus, computer equipment, and storage medium. Background Art

[0002] With the development of artificial intelligence technology, artificial intelligence-related pending data currently needs to be processed by a specific processor to perform data operations on the pending data. The most commonly used processor is the graphics processing unit (GPU), which can perform matrix operations, vector operations, etc. For example, the general matrix multiplication (Gemm) operation can be performed by the graphics processor.

[0003] In the related art, when a graphics processor performs general matrix multiplication on a matrix, a relatively efficient algorithm can be used to perform the operation. However, when a neural network processor (NPU) performs general matrix multiplication on a matrix, due to the difference in the computing units of the graphics processor and the neural network processor, the general matrix multiplication-related algorithms applicable to the graphics processor are not applicable to the neural network processor. Although some neural network processors come with related general matrix multiplication algorithms, these algorithms cannot effectively utilize the computing power of the neural network processor, resulting in a waste of computing power of the neural network processor.

[0004] Therefore, when the neural network processor in the related art performs general matrix multiplication operations, the computational efficiency is low, resulting in the inability to efficiently exert the computing power of the neural network processor. Summary of the Invention

[0005] The embodiments of the present application provide a data processing method, apparatus, computer equipment, and storage medium, which can improve the computational efficiency of a neural network processor when performing general matrix multiplication operations, thereby efficiently exerting the computational power of the neural network processor.

[0006] To achieve the above objectives, a first aspect of an embodiment of the present application provides a data processing method applied to a neural network processor, wherein the neural network processor includes a global memory, a transfer buffer area, and multiple computing units, the method comprising:

[0007] Performing block processing on the matrix to be updated to obtain multiple sub-matrices to be updated, and evenly distributing the multiple sub-matrices to be updated to each computing unit;

[0008] Determining a first submatrix corresponding to each submatrix to be updated in the first matrix, and moving the first submatrix from the global memory to a preset cache area of ​​a computing unit corresponding to each submatrix to be updated;

[0009] Determining, in the second matrix, a second submatrix corresponding to each of the submatrices to be updated, and transferring the second submatrix from the global memory to the transfer buffer area;

[0010] Obtaining a target first submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from the preset buffer area, and obtaining a target second submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from the transfer buffer area;

[0011] A general matrix multiplication operation is performed on each submatrix to be updated, the target first submatrix corresponding to each submatrix to be updated, and the target second submatrix according to the calculation unit corresponding to each submatrix to be updated to obtain an update submatrix corresponding to each submatrix to be updated, and an update matrix corresponding to the matrix to be updated is generated according to each update submatrix.

[0012] To achieve the above-mentioned object, a second aspect of an embodiment of the present application provides a data processing device, which is applied to a neural network processor. The neural network processor includes a global memory, a transfer buffer area, and multiple computing units. The device includes:

[0013] A block division module, configured to divide the matrix to be updated into blocks to obtain a plurality of sub-matrices to be updated, and evenly distribute the plurality of sub-matrices to be updated to each computing unit;

[0014] a first determining module, configured to determine, in the first matrix, a first submatrix corresponding to each submatrix to be updated, and transfer the first submatrix from the global memory to a preset buffer area of ​​a computing unit corresponding to each submatrix to be updated;

[0015] A second determining module is configured to determine, in the second matrix, a second submatrix corresponding to each submatrix to be updated, and transfer the second submatrix from the global memory to the transfer buffer area;

[0016] an acquisition module, configured to acquire, from the preset buffer area, a target first submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated, and acquire, from the transfer buffer area, a target second submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated;

[0017] a calculation module, configured to perform a general matrix multiplication operation on each submatrix to be updated, the target first submatrix corresponding to each submatrix to be updated, and the target second submatrix according to the calculation unit corresponding to each submatrix to be updated, to obtain an update submatrix corresponding to each submatrix to be updated, and to generate an update matrix corresponding to the matrix to be updated according to each update submatrix.

[0018] In some embodiments, the first determining module is configured to:

[0019] Obtain a first matrix, and perform block processing on the first matrix to obtain a first target matrix corresponding to each submatrix to be updated;

[0020] performing grouping processing on the first target matrix to obtain a first sub-matrix corresponding to each sub-matrix to be updated;

[0021] Each first sub-matrix is ​​moved from the global memory to a preset buffer area in the corresponding computing unit.

[0022] In some embodiments, the first determination module includes a first determination submodule, a blocking submodule, and a second determination submodule;

[0023] a first determining submodule, configured to determine a first threshold size of a matrix that can be stored in a first buffer of each computing unit, the first buffer being configured to obtain and store, from the preset cache area, a target first submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated;

[0024] A block submodule, configured to perform block processing on the first matrix according to the first threshold size to obtain a plurality of first sub-matrices;

[0025] The second determining submodule is configured to determine, from the plurality of first sub-matrices, a first sub-matrix corresponding to each of the sub-matrices to be updated.

[0026] In some embodiments, the second determining submodule is configured to:

[0027] Determining a first threshold size of a matrix that can be stored in a first buffer of each computing unit, the first buffer being used to obtain and store a target first submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from the preset cache area;

[0028] Divide the first matrix into blocks according to the first threshold size to obtain a plurality of first sub-matrices;

[0029] A first sub-matrix corresponding to each of the sub-matrices to be updated is determined from the multiple first sub-matrices.

[0030] In some embodiments, the first determining module is configured to:

[0031] Determining the capacity of a preset buffer area of ​​a computing unit corresponding to each sub-matrix to be updated;

[0032] Determining, according to the capacity and the first row and column information, a first sub-matrix that needs to be moved currently in the first sub-matrices that match the rows and columns of each sub-matrix to be updated;

[0033] The first sub-matrix that currently needs to be transferred is transferred from the global memory to a preset buffer area of ​​the calculation unit corresponding to each sub-matrix to be updated.

[0034] In some embodiments, the second determining module is configured to:

[0035] Determining a second threshold size of a matrix that can be stored in a second buffer of each computing unit, wherein the second buffer is used to obtain and store a target second sub-matrix requiring a general matrix multiplication operation corresponding to each sub-matrix to be updated from the transfer buffer area;

[0036] performing block processing on the second matrix according to the second threshold size to obtain a plurality of second sub-matrices;

[0037] A second sub-matrix corresponding to each of the sub-matrices to be updated is determined from the multiple second sub-matrices.

[0038] In some embodiments, the acquisition module is configured to:

[0039] Determining, in the preset buffer area, a candidate first sub-matrix corresponding to each sub-matrix to be updated;

[0040] Transferring the candidate first submatrix from the preset buffer area to a first buffer area of ​​a calculation unit corresponding to each submatrix to be updated, and obtaining a target first submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from the candidate first submatrix in the first buffer area;

[0041] Determining, in the transfer buffer area, a candidate second submatrix corresponding to each submatrix to be updated;

[0042] The candidate second submatrix is ​​transferred to a second buffer of a calculation unit corresponding to each submatrix to be updated, and a target second submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated is obtained from the candidate second submatrix in the second buffer.

[0043] In some embodiments, the first buffer area includes a first sub-area and a second sub-area, the second buffer area includes a third sub-area and a fourth sub-area, and the first determining module is configured to:

[0044] Determine a first idle sub-area in the first sub-area and the second sub-area;

[0045] Moving the candidate first sub-matrix from the preset buffer area to the first idle sub-area;

[0046] The second determining module is configured to:

[0047] determining a second idle sub-area in the third sub-area and the fourth sub-area;

[0048] The candidate second sub-matrix is ​​moved from the preset buffer area to the second idle sub-region.

[0049] In some embodiments, the computing module is configured to:

[0050] Inputting the target first submatrix and the target second submatrix into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated to perform a matrix multiplication operation to obtain a target submatrix corresponding to each submatrix to be updated;

[0051] Each of the sub-matrices to be updated and the target sub-matrix is ​​input into a vector calculation unit of a calculation unit corresponding to each of the sub-matrices to be updated, and a vector addition operation is performed to obtain an updated sub-matrix corresponding to each of the sub-matrices to be updated.

[0052] In some embodiments, after inputting the target first submatrix and the target second submatrix into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated to perform a matrix multiplication operation to obtain the target submatrix corresponding to each submatrix to be updated, the calculation module is configured to:

[0053] Deleting the target first sub-matrix from the first buffer to update a first idle sub-region in the first buffer;

[0054] The target second sub-matrix is ​​deleted from the second buffer to update a second idle sub-region in the second buffer.

[0055] In some embodiments, the segmentation module is configured to:

[0056] determining a total number of the plurality of computing units;

[0057] The multiple matrices to be updated are evenly distributed to each computing unit according to the total number.

[0058] In order to achieve the above-mentioned purpose, the third aspect of an embodiment of the present application provides a computer-readable storage medium, which stores multiple instructions, and the instructions are suitable for a processor to load to execute the data processing method provided by the embodiment of the present application.

[0059] In order to achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the data processing method provided in the embodiment of the present application when executing the computer program.

[0060] In an embodiment of the present application, a neural network processor includes a global memory, a transfer buffer area and multiple computing units. A plurality of sub-matrices to be updated are obtained by performing block processing on a matrix to be updated, and the plurality of sub-matrices to be updated are evenly distributed to each computing unit; a first sub-matrix corresponding to each sub-matrix to be updated is determined in a first matrix, and the first sub-matrix is ​​moved from the global memory to a preset buffer area of ​​a computing unit corresponding to each sub-matrix to be updated; a second sub-matrix corresponding to each sub-matrix to be updated is determined in a second matrix, and the second sub-matrix is ​​moved from the global memory to the transfer buffer area; a target first sub-matrix requiring a general matrix multiplication operation corresponding to each sub-matrix to be updated is obtained from the preset buffer area, and a target second sub-matrix requiring a general matrix multiplication operation is obtained from the transfer buffer area for each sub-matrix to be updated; a general matrix multiplication operation is performed on each sub-matrix to be updated, a target first sub-matrix corresponding to each sub-matrix to be updated and a target second sub-matrix corresponding to each sub-matrix to be updated according to the computing unit corresponding to each sub-matrix to be updated, to obtain an updated sub-matrix corresponding to each sub-matrix to be updated, and an updated matrix corresponding to the matrix to be updated is generated according to each updated sub-matrix.

[0061] In this way, by performing block processing on the matrix to be updated, multiple sub-matrices to be updated are obtained, and the first matrix and the second matrix are processed in blocks to obtain the first sub-matrix and the second sub-matrix corresponding to each sub-matrix to be updated, and then the first sub-matrix and the second sub-matrix that each computing unit needs to process are moved from the global memory to the corresponding cache area in advance. When each computing unit performs a general matrix multiplication operation subsequently, the number of times data is moved from the global memory can be reduced, and each computing unit processes the evenly distributed sub-matrix to be updated, the first sub-matrix and the second sub-matrix, which can achieve load balancing when multiple computing units perform general matrix multiplication operations, thereby improving the efficiency of each computing unit when performing general matrix multiplication operations, thereby efficiently exerting the computing power of the neural network processor.

[0062] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present application. The purposes and other advantages of the present application can be achieved and obtained through the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0064] Figure 1 is a schematic diagram of the structure of a neural network processor provided in an embodiment of the present application;

[0065] Figure 2 is a schematic diagram of the structure of a computing unit provided in an embodiment of the present application;

[0066] Figure 3 This is a flow chart of a data processing method provided in an embodiment of the present application;

[0067] Figure 4 This is a schematic diagram of matrix block processing provided by an embodiment of the present application;

[0068] Figure 5 is a schematic diagram of matrix grouping provided in an embodiment of the present application;

[0069] Figure 6 This is another schematic diagram of matrix grouping provided by an embodiment of the present application;

[0070] Figure 7 This is another schematic diagram of matrix grouping provided by an embodiment of the present application;

[0071] Figure 8 This is a schematic diagram of the structure of the first buffer zone and the second buffer zone provided in an embodiment of the present application;

[0072] Figure 9 is another flow chart of the data processing method provided in an embodiment of the present application;

[0073] Figure 10 It is a structural diagram of a data processing device implemented in this application;

[0074] Figure 11 It is a structural diagram of the computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0075] In order to enable those skilled in the art to better understand the solutions of this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of this application.

[0076] It is understandable that in the specific implementation of this application, the matrix-related data involved, when the above embodiments of this application are applied to specific products or technologies, need to obtain user permission or consent, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards.

[0077] It should be noted that some processes described in the specification, claims and the above-mentioned drawings include multiple steps that appear in a specific order, but it should be clearly understood that these steps may not be executed in the order in which they appear in this document or may be executed in parallel. The step numbers are only used to distinguish between different steps, and the numbers themselves do not represent any execution order.

[0078] Before further explaining the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations:

[0079] A neural network processor (NPU) is a hardware device specifically designed to accelerate neural network computations. Specifically, it can perform matrix, vector, and scalar calculations. It can perform general matrix multiplication operations between matrices. It contains multiple computational units, each capable of performing general matrix multiplication operations between matrices.

[0080] General Matrix Multiplication (GEMM) is a fundamental linear algebra operation used to compute the product of two matrices. It is a highly versatile computational operation that can be applied to a wide variety of fields and problems. Many scientific computing, engineering applications, and machine learning algorithms involve matrix multiplication, making efficient implementations crucial for performance in these fields. Because matrix multiplication is often computationally intensive, its performance significantly impacts the performance of the entire computing system. Optimizing the implementation of GEMM can significantly improve computational efficiency, reducing computation time and resource consumption.

[0081] The above is a detailed explanation of the relevant technical terms.

[0082] First, let’s explain the technical problems existing in related technologies:

[0083] With the development of artificial intelligence technology, artificial intelligence-related pending data currently needs to be processed by a specific processor to perform data operations on the pending data. The most commonly used processor is the graphics processing unit (GPU), which can perform matrix operations, vector operations, etc. For example, the general matrix multiplication (Gemm) operation can be performed by the graphics processor.

[0084] In the related art, when a graphics processor performs general matrix multiplication on a matrix, a relatively efficient algorithm can be used to perform the operation. However, when a neural network processor (NPU) performs general matrix multiplication on a matrix, due to the difference in the computing units of the graphics processor and the neural network processor, the general matrix multiplication-related algorithms applicable to the graphics processor are not applicable to the neural network processor. Although some neural network processors come with related general matrix multiplication algorithms, these algorithms cannot effectively utilize the computing power of the neural network processor, resulting in a waste of computing power of the neural network processor.

[0085] Therefore, when the neural network processor in the related art performs general matrix multiplication operations, the computational efficiency is low, resulting in the inability to efficiently exert the computing power of the neural network processor.

[0086] Take the Ascend processor, a neural network processor, for example. Its computational units differ from those of traditional graphics processors, so general matrix multiplication algorithms suitable for GPUs are inapplicable to this processor. While this processor does have its own general matrix multiplication algorithms, they are unable to fully utilize the processor's computing power.

[0087] For example, in this neural network processor, the size of input matrix A is m*k, the size of input matrix B is k*n, and the size of input and output matrix C is m*n. It is known that the theoretical peak performance of FP16 of this neural network processor is 313TFlops, which means that it can perform 313*10^12 floating-point calculations per second. After inputting the relevant matrices into the neural network processor, the test results are shown in Table 1:

[0088] Test matrix size Test performance (TFlops) m=3200k=3200n=3200 162.6 m=6400k=6400n=6400 196.9 m=12800k=12800n=12800 164.5 m=25600k=25600n=25600 171.2

[0089] Table 1

[0090] As shown in Table 1, the best tested performance is 196.9 TFlops, while the theoretical peak performance of the neural network processor is 313 TFlops, resulting in a floating point utilization rate of approximately 196.9 / 313*100% = 62.9%. This means that approximately 37% of the computing power is not utilized during general matrix multiplication operations, resulting in low efficiency for the neural network processor.

[0091] To this end, embodiments of the present application provide a data processing method suitable for a neural network processor, which can improve the computational efficiency of the neural network processor when performing general matrix multiplication operations, thereby effectively unleashing the computing power of the neural network processor. The data processing method, apparatus, computer device, and storage medium provided in embodiments of the present application are described in detail below.

[0092] See also Figure 1 , Figure 1 Schematic diagram of the structure of the neural network processor provided in the embodiment of the present application.

[0093] like Figure 1 As shown in the figure, the neural network processor includes multiple computing units, which can perform computing tasks in parallel, such as performing general matrix multiplication on matrices. Each computing unit is connected to global memory, and each computing unit can obtain matrix data and transfer calculation results to global memory, such as the result of general matrix multiplication.

[0094] In some embodiments, each computing unit includes units with different functions, such as a matrix computing unit, a vector computing unit, and a scalar computing unit. These units with different functions can implement related functions. For example, the matrix computing unit is responsible for performing matrix operations, the vector computing unit is responsible for performing vector operations, and the scalar computing unit is responsible for various types of scalar data operations and program flow control.

[0095] Please combine Figure 2 , Figure 2 It is a structural diagram of the computing unit provided in an embodiment of the present application.

[0096] The calculation unit includes a first buffer, a second buffer, a third buffer, a preset buffer area, a matrix calculation unit, a vector calculation unit and a unified buffer.

[0097] Among them, the storage areas such as the first buffer area, the second buffer area, the third buffer area and the preset cache area are mainly related to the matrix calculation unit, and the matrix calculation unit stores or reads data in these storage areas.

[0098] For example, matrix data can be pre-stored in a preset cache area. The preset cache area acts as a data transfer area and can temporarily store matrix data for subsequent matrix calculation units to use for matrix operations. For example, matrix data can be obtained from the global memory of the neural network processor and moved to the preset cache area.

[0099] The first buffer can be used to store matrix data that the matrix calculation unit is about to need matrix operation. The matrix data that is about to need matrix operation can be moved from the preset cache area to the first buffer, and then the matrix calculation unit can obtain the matrix data that is about to need matrix operation from the first buffer.

[0100] The second buffer can be used to store another matrix data that the matrix calculation unit is about to need for matrix operation. The other matrix data that is about to need for matrix operation can be moved from the global memory or the transfer cache to the second buffer, and then the matrix calculation unit can obtain the other matrix data that is about to need for matrix operation from the second buffer.

[0101] The matrix calculation unit can perform a matrix operation on the matrix data requiring matrix operation and another matrix data, such as a matrix multiplication operation, to obtain a matrix operation result, and then send the matrix operation result to the third buffer.

[0102] The third buffer can store the matrix operation results output by the matrix calculation unit after performing the matrix operation, and can send the matrix operation results to the global memory for storage.

[0103] Global memory can be understood as a storage area accessed by all computing units in a neural network processor. It typically has a relatively large storage capacity and can store large amounts of data. This makes it suitable for storing entire datasets or large model parameters. Global memory can be accessed by different computing units at any time during program execution. Access latency for global memory is high because data must be read from a relatively distant storage location, involving transfers across the memory bus and a lookup in the storage hierarchy.

[0104] The transfer buffer can be understood as a cache area independent of global memory and the computational unit. Its function is to reduce the number of times the computational unit accesses global memory, thereby improving the computational efficiency of the computational unit. For example, the matrix data corresponding to the matrix computation unit can be stored in the transfer buffer, and at least part of the matrix data can be subsequently transferred from the transfer buffer to the secondary buffer.

[0105] A unified buffer is a specific area or structure used to store and manage data, which can be shared and accessed by multiple components or programs. The purpose of a unified buffer is to improve data storage and access efficiency, reduce data copying and movement, and thus optimize system performance. For example, vectors and matrices required for calculations by a vector computing unit can be moved from global memory to a unified buffer.

[0106] The vector calculation unit can obtain the matrices and vectors required for vector operations from the uniform buffer, and then perform vector calculations, such as vector addition operations, and finally send the vector calculation results to the uniform buffer, and then send the calculation results to the global memory through the uniform buffer.

[0107] The above is an introduction to the entire computing unit. The data processing method provided by the embodiment of the present application will be described in detail below.

[0108] The data processing methods provided in this application can be used in a variety of general or special computer system environments or configurations. For example: personal computers, server computers, multi-processor systems, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0109] See 3, Figure 3 : is a flow chart of a data processing method provided in an embodiment of the present application. The data processing method may include the following steps:

[0110] Step 110: Divide the matrix to be updated into blocks to obtain multiple sub-matrices to be updated, and evenly distribute the multiple sub-matrices to be updated to each computing unit;

[0111] Step 120: Determine in the first matrix a first submatrix corresponding to each submatrix to be updated, and move the first submatrix from the global memory to a preset buffer area of ​​a computing unit corresponding to each submatrix to be updated;

[0112] Step 130: Determine in the second matrix the second sub-matrix corresponding to each sub-matrix to be updated, and move the second sub-matrix from the global memory to the transfer buffer area;

[0113] Step 140: Obtain a target first submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from a preset buffer, and obtain a target second submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from a transfer buffer;

[0114] Step 150: Perform a general matrix multiplication operation on each submatrix to be updated, a target first submatrix corresponding to each submatrix to be updated, and a target second submatrix according to the calculation unit corresponding to each submatrix to be updated to obtain an update submatrix corresponding to each submatrix to be updated, and generate an update matrix corresponding to the matrix to be updated according to each update submatrix.

[0115] Steps 110 to 150 will be described in detail below.

[0116] In step 110, the matrix to be updated is divided into blocks to obtain a plurality of sub-matrices to be updated, and the plurality of sub-matrices to be updated are evenly distributed to each computing unit.

[0117] In this application, the neural network processing has multiple computing units, and multiple computing units can perform calculations simultaneously. Therefore, the matrix to be updated can be divided into multiple sub-matrices to be updated, and then the multiple sub-matrices to be updated are evenly distributed to different computing units. Each computing unit corresponds to at least one sub-matrix to be updated, and each computing unit can process the corresponding sub-matrix to be updated. This achieves load balancing among multiple computing units.

[0118] Each computational unit can then update the corresponding submatrix to be updated, such as through a general matrix multiplication operation, to obtain the updated submatrix. Finally, the update matrix corresponding to the matrix to be updated is generated based on each update submatrix. Multiple computational units perform calculations simultaneously, achieving load balancing across multiple computational units and fully utilizing the computing power of each computational unit. This greatly reduces the time required to update the matrix to be updated and more efficiently utilizes the computing power of the neural network processing.

[0119] It should be noted that when the update matrix is ​​divided into blocks, the size of the sub-matrix to be updated needs to be taken into consideration. For example, a preset size can be set, and the size of the sub-matrix to be updated should be smaller than the preset size. This can prevent the size of the sub-matrix to be updated from being too large, resulting in an excessive amount of data in the sub-matrix to be updated. The update sub-matrix needs to continue to be divided into blocks, thereby improving the efficiency of the block division.

[0120] In some implementations, evenly distributing multiple sub-matrices to be updated to each computing unit includes:

[0121] (1.1) Determine the total number of multiple computing units;

[0122] (1.2) Multiple matrices to be updated are evenly distributed to each computing unit according to the total number.

[0123] The total number of computing units in the neural network processor can be determined, and then the sub-matrices to be updated can be evenly distributed to each computing unit. For example, the same number of sub-matrices to be updated can be distributed equally to each computing unit, or the difference in the number of sub-matrices to be updated distributed to any two computing units is within a preset range. This can achieve load balancing for each computing unit during calculations.

[0124] For a more detailed understanding of the process of updating matrix blocks in this application, please refer to Figure 4 , Figure 4 This is a schematic diagram of matrix block processing provided by an embodiment of the present application.

[0125] In this application, the calculation formula corresponding to each computing unit when performing a general matrix multiplication operation is: C←αAB+βC. Where A is the first matrix, B is the second matrix, C on the right side of the formula is the matrix to be updated, C on the left side of the formula is the update matrix, α is the first preset floating-point number, and β is the second preset floating-point number.

[0126] like Figure 4 As shown, the size of the first matrix is ​​M*K, the size of the second matrix is ​​K*N, and the size of the matrix to be updated is M*N.

[0127] The general matrix multiplication formula shows that updating the matrix to be updated requires combining the first and second matrices. Therefore, the matrix to be updated must be divided into blocks to obtain multiple sub-matrices to be updated. Simultaneously, the first matrix must be divided into blocks to obtain a first sub-matrix corresponding to each sub-matrix to be updated. The second matrix must also be divided into blocks to obtain a second sub-matrix corresponding to each sub-matrix to be updated. Finally, each sub-matrix to be updated is updated using the first and second sub-matrices corresponding to each sub-matrix to be updated.

[0128] First, the block processing of the matrix to be updated is introduced, and then how the first matrix and the second matrix are processed in blocks is introduced.

[0129] The matrix to be updated is a large matrix, and a partitioning standard can be set within a preset size. For example, the partitioning standard is an M1*N1 size standard. The matrix to be updated can be divided into blocks according to this standard, and the size of most of the sub-matrices to be updated is M1*N1.

[0130] Under this division standard, there may be a situation where the matrix to be updated cannot be evenly divided, and there may be some sub-matrices with sizes smaller than M1*N1. These sub-matrices are also determined as sub-matrices to be updated. Figure 4As shown, the size of the rightmost and bottommost submatrices to be updated is smaller than M1*N1.

[0131] The general matrix multiplication formula shows that updating the matrix to be updated requires combining the first and second matrices. Therefore, after partitioning the matrix to be updated to obtain multiple sub-matrices to be updated, the first matrix must also be partitioned to obtain the first sub-matrix corresponding to each sub-matrix to be updated. The second matrix is ​​then partitioned to obtain the second sub-matrix corresponding to each sub-matrix to be updated. Finally, each sub-matrix to be updated is updated using the corresponding first and second sub-matrices.

[0132] In step 120 , a first sub-matrix corresponding to each sub-matrix to be updated is determined in the first matrix, and the first sub-matrix is ​​moved from the global memory to a preset buffer area of ​​a computing unit corresponding to each sub-matrix to be updated.

[0133] It is understandable that after the matrix to be updated is partitioned, the first matrix also needs to be partitioned to obtain a first submatrix corresponding to the submatrix to be updated in the first matrix. The first submatrix is ​​then moved from the global memory to a preset cache area of ​​the computing unit corresponding to each submatrix to be updated. When the computing unit subsequently performs a general matrix multiplication operation, it can obtain the first submatrix from the preset cache area, thereby reducing the number of interactions with the global memory.

[0134] Global memory stores large amounts of data and typically uses relatively slow storage media. This results in a significant time spent searching for data in global memory, and additional data retrieval from global memory wastes search time. The preset cache, however, is a high-speed internal memory within the computational unit. Pre-fetching the first submatrix to the preset cache allows subsequent computational units to quickly retrieve the first submatrix when performing general matrix multiplication operations. This also reduces the number of interactions with global memory, thereby improving the computational efficiency of each computational unit.

[0135] In some implementations, determining in the first matrix a first submatrix corresponding to each submatrix to be updated includes:

[0136] (1.1) determining a first threshold size of a matrix that can be stored in a first buffer of each computing unit, the first buffer being used to obtain and store a target first submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from a preset cache area;

[0137] (1.2) dividing the first matrix into blocks according to a first threshold size to obtain a plurality of first sub-matrices;

[0138] (1.3) Determine a first submatrix corresponding to each submatrix to be updated from the multiple first submatrices.

[0139] Among them, each computing unit also includes a first buffer, which is used to obtain and store the target first sub-matrix requiring general matrix multiplication operation corresponding to each sub-matrix to be updated from a preset cache area. It can also be understood that the first sub-matrix that the computing unit is about to perform general matrix multiplication operation is moved from the preset cache area to the first buffer in advance. The matrix calculation unit in the subsequent computing unit can directly obtain the target first sub-matrix requiring general matrix multiplication operation from the first buffer, which can further improve the computing efficiency of the computing unit.

[0140] However, the memory of the first buffer is limited, so it is necessary to determine a first threshold size of the matrix that the first buffer can store. The first threshold size is the size of the largest matrix that the first buffer can cache. Therefore, the size of the first submatrix transferred to the first buffer must be smaller than the first threshold size.

[0141] In the present application, a dual storage area is set for the first buffer. The first sub-matrix in one storage area can be read and calculated by the matrix calculation unit, and the first sub-matrix in the other storage area can wait for the next reading and calculation by the matrix calculation unit. After the first sub-matrix in a storage area of ​​the dual storage area is read and calculated, the data in the storage area can be deleted, and then a new first sub-matrix can be obtained from the preset cache area and stored in the storage area.

[0142] Therefore, in the present application, the first matrix can be divided into blocks according to the first threshold size to obtain multiple first sub-matrices, so that the first sub-matrices meet the dual storage area requirements of the first buffer, for example, the sum of the sizes of the two first sub-matrices after addition is smaller than the first threshold size.

[0143] Combine Figure 4 In this application, the size division standard of the first submatrix can be M1*K1. The size of most of the first submatrices obtained is M1*K1. Under this division standard, there may be cases where the first matrix cannot be evenly divided, and some submatrices may have sizes smaller than M1*K1. These submatrices are also determined as first submatrices. Figure 4 As shown, the size of the first rightmost and bottommost submatrices is smaller than M1*K1.

[0144] In the present application, multiple sub-matrices to be updated, multiple first sub-matrices, and multiple second sub-matrices can be grouped. For example, the total number of multiple computing units is used as a grouping criterion. Assuming that the total number is 3, the multiple sub-matrices to be updated can be divided into multiple groups according to the grouping criterion. When the subsequent computing unit updates the sub-matrix to be updated, the sub-matrix to be updated of each group can be updated in sequence, that is, after updating the sub-matrix to be updated of the previous group, the sub-matrix to be updated of the next group is updated.

[0145] Please combine Figure 5 , Figure 5 This is a schematic diagram of matrix grouping provided in an embodiment of the present application.

[0146] For example, the total number of computing units is used as the grouping criterion. Assuming the total number is 3, the matrix rows are divided to obtain multiple groups corresponding to the sub-matrices to be updated. After the sub-matrices to be updated are grouped, the first sub-matrix and the second sub-matrix need to be grouped according to the grouping of the sub-matrices to be updated.

[0147] Taking a group C1 of submatrices to be updated as an example, the submatrices to be updated of group C1 correspond to group A1 in the first matrix, that is, all first submatrices in group A1 are related to all submatrices to be updated in group C1. The submatrices to be updated of group C1 correspond to groups B1, B2, and B3 in the second matrix, that is, all second submatrices in groups B1, B2, and B3 are related to all submatrices to be updated in group C1. The above-described groups are Figure 5 The middle is the dark area with color.

[0148] Furthermore, since the capacity of the preset buffer area of ​​the computing unit is limited, and the first sub-matrix needs to be pre-stored in the preset buffer area later, a large number of first sub-matrices cannot be stored in the preset buffer area at one time, then the grouping process can be continued for the group corresponding to the first matrix to obtain multiple sub-groups, such as Figure 5 As shown, group A1 is further grouped to obtain subgroups A11, A12, and A13. From each of subgroups A11, A12, and A13, the first submatrix corresponding to each matrix to be updated can be selected and transferred to the preset buffer area of ​​the calculation unit corresponding to each matrix to be updated.

[0149] In the present application, the grouping of multiple sub-matrices to be updated, the grouping of multiple first sub-matrices, and the grouping of multiple second sub-matrices can be performed according to matrix rows or matrix columns, and each sub-matrix to be updated has corresponding row and column information in the entire matrix to be updated.

[0150] exist Figure 5In the grouping shown, the multiple sub-matrices to be updated, the multiple first sub-matrices, and the multiple second sub-matrices are grouped by matrix rows. Each of the multiple sub-matrices to be updated has corresponding row and column information. Each of the multiple first sub-matrices has corresponding row and column information. Each of the multiple second sub-matrices has corresponding row and column information. The row and column information can specifically be the row and column number, which can be used to locate a specific sub-matrix.

[0151] Please also refer to Figure 6 and Figure 7 , Figure 6 This is another schematic diagram of matrix grouping provided in an embodiment of the present application. Figure 7 This is another schematic diagram of matrix grouping provided in an embodiment of the present application.

[0152] As can be seen from the above content, in the embodiment of the present application, when the update matrix, the first matrix, and the second matrix are block-processed, there will be some sub-matrices whose sizes are smaller than the normal block size. These sub-matrices with smaller sizes can be grouped together with adjacent sub-matrices of normal size.

[0153] like Figure 6 and Figure 7 As shown, in the matrix to be updated, the sub-matrix to be updated with a smaller size and the adjacent sub-matrix to be updated with a normal size are grouped together, and this part of the matrix is ​​the remaining matrix.

[0154] Specifically, the remaining matrices can be grouped in a matrix column manner to obtain group ZC1 and group ZC2. Similarly, by matching the row and column information of the sub-matrix to be updated, the first value matrix, and the second sub-matrix, it is determined that the groups corresponding to group ZC1 are groups XA1, XA2, XA3, YB1, YB2, and YB3. That is to say, the first sub-matrix and the second sub-matrix corresponding to the sub-matrix to be updated in group ZC1 can be determined from these groups.

[0155] It is determined that the groups corresponding to group ZC2 include groups XA1, XA2, XA3, YB4, YB5, and YB6. That is, the first sub-matrix and the second sub-matrix corresponding to the sub-matrix to be updated in group ZC2 can be determined from these groups.

[0156] As can be seen from the above content, in the present application, the sub-matrix to be updated, the first sub-matrix, and the second sub-matrix can be associated through row and column information to generate a mapping relationship corresponding to the row and column information, so that when a sub-matrix to be updated is updated, the first sub-matrix and the second sub-matrix that match the sub-matrix to be updated can be determined according to the mapping relationship.

[0157] In some embodiments, a group association relationship can be set between the group of the matrix to be updated, the group of the first matrix, and the group of the second matrix. For example, group C1 corresponds to group A1 and groups B1, B2, and B3. Then, a mapping relationship of row and column information between the sub-matrices in group C1, group A1, and groups B1, B2, and B3 is established. When a sub-matrix to be updated is subsequently updated, the group associated with the sub-matrix to be updated in the first matrix and the second matrix can be searched based on the focus association relationship. Then, based on the mapping relationship of row and column information, the first sub-matrix and the second sub-matrix that match the sub-matrix to be updated can be found in the associated group. This can improve the efficiency of sub-matrix search, thereby increasing the speed of sub-matrix search, and ultimately improving the computational efficiency of the computing unit.

[0158] In some implementations, determining a first submatrix corresponding to each submatrix to be updated from a plurality of first submatrices includes:

[0159] (1.3.1) determining target row and column information of each to-be-updated submatrix in the to-be-updated matrix, and determining first row and column information of each first submatrix in the first matrix among the plurality of first submatrices;

[0160] (1.3.2) According to the first row and column information, determine a first sub-matrix from the plurality of first sub-matrices that matches the target row and column information of each sub-matrix to be updated.

[0161] Among them, from the above content, it can be known that each sub-matrix to be updated has corresponding target row and column information in the matrix to be updated, Figure 5 Taking the first submatrix to be updated in the upper left corner of the matrix to be updated as an example, its corresponding row is the first row, and its corresponding column is the first column, so the target row and column information is the first row and first column.

[0162] Each of the multiple first submatrices also has corresponding first row and column information. The rows of each first submatrix are determined using the rows of the target row and column information and the first row and column information. Row matching is then performed on the two, thereby determining that the multiple first submatrices in the first row of the first matrix are the first submatrices that match the submatrix to be updated.

[0163] After determining the first submatrix that matches the target row and column information for each submatrix to be updated, the first submatrix can be moved from global memory to the preset cache area of ​​the calculation unit corresponding to each submatrix to be updated. This allows subsequent calculation units to directly access the first submatrix from the preset cache area when performing calculations, rather than having to retrieve the corresponding first submatrix from global memory each time a calculation is performed. This improves the computational efficiency of the calculation units.

[0164] In some embodiments, moving the first sub-matrix from the global memory to a preset cache area of ​​a computing unit corresponding to each sub-matrix to be updated includes:

[0165] (2.1) Determine the capacity of the preset buffer area of ​​the computing unit corresponding to each sub-matrix to be updated;

[0166] (2.2) According to the capacity and the first row and column information, determine the first submatrix that needs to be moved currently in the first submatrix that matches the rows and columns of each submatrix to be updated;

[0167] (2.3) The first sub-matrix that needs to be moved is moved from the global memory to the preset buffer area of ​​the calculation unit corresponding to each sub-matrix to be updated.

[0168] Among them, the capacity of the preset cache area of ​​each computing unit is limited. For example, the preset cache area can only cache three first sub-matrices at a time. Therefore, the number of first sub-matrices transferred from the global memory to the preset cache area can be determined according to the capacity of the preset cache area.

[0169] After determining the number of transfers, the row and column information of each submatrix to be updated and the first row and column information of multiple first submatrices can be matched. As can be seen above, the matching can be performed based on the mapping relationship between the row and column information, thereby determining the first submatrix that matches each submatrix to be updated. These first submatrices are the first submatrices that need to be transferred currently, and the number of first submatrices is the same as the number of transfers.

[0170] The first submatrix that needs to be moved is then moved from global memory to a preset cache area of ​​each computing unit corresponding to the submatrix to be updated. When the computing unit subsequently obtains the first submatrix for calculation, it can obtain the target first submatrix that requires a general matrix multiplication operation from the preset cache area, thereby reducing the number of times the target first submatrix that requires a general matrix multiplication operation is obtained from global memory. This can improve the computational efficiency of the computing unit.

[0171] Please combine Figure 5 Assuming the preset cache can only cache three first sub-matrices at a time, the number of matrices to be moved is 3. The sub-matrix to be updated is the sub-matrix in the first row and first column. The corresponding first sub-matrix is ​​the first sub-matrix in the first row of the first matrix. The first sub-matrix in the first row has three groups. First, the three first sub-matrices in the first row of group A11 can be moved from global memory to the preset cache. After the first sub-matrices in the preset cache are moved and read, the first sub-matrix in the first row of group A12 can be moved to the preset cache, and the next round of general matrix multiplication can begin.

[0172] It should be noted that, assuming there are three computing units, the submatrix to be updated in the first row of group C1 can be assigned to the first computing unit, the submatrix to be updated in the second row of group C1 can be assigned to the second computing unit, and the submatrix to be updated in the third row of group C1 can be assigned to the third computing unit. This can achieve load balancing across multiple computing units and further improve computing efficiency.

[0173] After all the sub-matrices to be updated in group C1 are updated, the sub-matrix to be updated in the first row of group C2 can be assigned to the first calculation unit, the sub-matrix to be updated in the second row of group C2 can be assigned to the second calculation unit, and the sub-matrix to be updated in the third row of group C2 can be assigned to the third calculation unit. In this way, the sub-matrix to be updated in each group can be updated sequentially.

[0174] In step 130 , a second sub-matrix corresponding to each sub-matrix to be updated is determined in the second matrix, and the second sub-matrix is ​​moved from the global memory to the transfer buffer.

[0175] It can be seen from the above general matrix multiplication operation that a second matrix is ​​required to update the matrix to be updated. For the update of each sub-matrix to be updated, the update can be achieved through the first sub-matrix and the second sub-matrix of each sub-matrix to be updated.

[0176] The second matrix can be processed in blocks, so as to obtain the second sub-matrix corresponding to the sub-matrix to be updated in the first matrix. The second sub-matrix is ​​then moved from the global memory to the transfer cache. When the subsequent computing unit performs a general matrix multiplication operation, it can obtain the second sub-matrix from the transfer cache, thereby reducing the number of interactions with the global memory. Compared with the global memory, the transfer cache has a higher transmission speed and lower latency. Therefore, the second sub-matrix is ​​pre-stored in the transfer cache. When the computing unit performs a matrix multiplication operation, it can quickly obtain the second sub-matrix, thereby improving computing efficiency.

[0177] In some implementations, determining the second submatrix corresponding to each submatrix to be updated in the second matrix includes:

[0178] (1.1) determining a second threshold size of a matrix that can be stored in a second buffer of each computing unit, the second buffer being used to obtain and store a target second submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from the transfer buffer;

[0179] (1.2) dividing the second matrix into blocks according to a second threshold size to obtain a plurality of second sub-matrices;

[0180] (1.3) Determine, from the plurality of second sub-matrices, a second sub-matrix corresponding to each sub-matrix to be updated.

[0181] Among them, each computing unit also includes a second buffer, which is used to obtain and store the target second sub-matrix that requires general matrix multiplication operation corresponding to each sub-matrix to be updated from a preset buffer area. It can also be understood that the second sub-matrix that the computing unit is about to perform general matrix multiplication calculation is moved from the transfer buffer area to the second buffer in advance. The matrix calculation unit in the subsequent computing unit can directly obtain the target second sub-matrix that requires general matrix multiplication operation from the second buffer, which can further improve the computing efficiency of the computing unit.

[0182] However, the memory of the second buffer is limited, so it is necessary to determine a second threshold size of the matrix that the second buffer can store. The second threshold size is the size of the largest matrix that the second buffer can cache. Therefore, the size of the second submatrix transferred to the second buffer must be smaller than the second threshold size.

[0183] In the present application, a dual storage area is set for the second buffer. The second sub-matrix in one storage area can be read and calculated by the matrix calculation unit, and the second sub-matrix in the other storage area can wait for the next reading and calculation by the matrix calculation unit. After the second sub-matrix in a storage area of ​​the dual storage area is read and calculated, the data in the storage area can be deleted, and then a new second sub-matrix can be obtained from the preset cache area and stored in the storage area.

[0184] Therefore, in the present application, the second matrix can be divided into blocks according to the second threshold size to obtain multiple second sub-matrices, so that the second sub-matrices meet the dual storage area requirements of the first buffer, for example, the sum of the sizes of the two second sub-matrices after addition is less than the second threshold size.

[0185] Finally, the target row and column information of each submatrix to be updated in the matrix to be updated is determined, and the second row and column information of each second submatrix in the second matrix is ​​determined. Based on the second row and column information, a second submatrix that matches the target row and column information of each submatrix to be updated is determined in the plurality of second submatrices. For example, combined with Figure 5 Assuming the target row and column information of the submatrix to be updated is the first row and first column of the matrix to be updated, all second submatrices in the first column of the second matrix are determined as the second submatrix corresponding to the matrix to be updated. The second submatrix is ​​then moved from the global memory to the transfer buffer.

[0186] In step 140, a target first submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated is obtained from a preset buffer area, and a target second submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated is obtained from a transfer buffer area.

[0187] In the present application, the first submatrix corresponding to the submatrix to be updated is pre-stored in a preset cache area, and the second submatrix corresponding to the submatrix to be updated is pre-stored in a transfer cache area. When the subsequent calculation unit performs calculations, it can obtain the target first submatrix corresponding to each submatrix to be updated that requires a general matrix multiplication operation from the preset cache area, and obtain the target second submatrix corresponding to each submatrix to be updated that requires a general matrix multiplication operation from the transfer cache area.

[0188] In some embodiments, obtaining a target first submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from a preset buffer, and obtaining a target second submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from a transfer buffer, includes:

[0189] (1.1) determining, in a preset buffer area, a candidate first submatrix corresponding to each submatrix to be updated;

[0190] (1.2) moving the candidate first submatrices from the preset buffer area to the first buffer area of ​​the computing unit corresponding to each submatrix to be updated, and obtaining the target first submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated from the candidate first submatrix in the first buffer area;

[0191] (1.3) determining in the transfer buffer area a candidate second submatrix corresponding to each submatrix to be updated;

[0192] (1.4) The candidate second submatrix is ​​transferred to the second buffer of the computing unit corresponding to each submatrix to be updated, and the target second submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated is obtained from the candidate second submatrix in the second buffer.

[0193] Among them, there are multiple first sub-matrices in the preset cache area. The multiple first sub-matrices can be sorted first to obtain a sorting result, and then the first sub-matrix to be calculated and used by the matrix calculation unit is determined according to the sorting result, and then these first sub-matrices are determined as candidate first sub-matrices.

[0194] The candidate first submatrix is ​​moved from a preset buffer area to a first buffer area of ​​a computing unit corresponding to each submatrix to be updated. In the first buffer area, there may be multiple candidate first submatrices. From the multiple candidate first submatrices, a target first submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated may be selected based on a sorting result.

[0195] There are multiple second sub-matrices in the transfer buffer area. The multiple second sub-matrices can be sorted first to obtain a sorting result, and then the second sub-matrices to be calculated and used by the matrix calculation unit are determined according to the sorting result, and then these second sub-matrices are determined as candidate second sub-matrices.

[0196] The candidate first submatrix is ​​moved from a preset buffer area to a second buffer area of ​​a computing unit corresponding to each submatrix to be updated. In the second buffer area, there may be multiple candidate second submatrices. From the multiple candidate second submatrices, a target second submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated may be selected based on a sorting result.

[0197] Please combine Figure 8 , Figure 8 This is a schematic diagram of the structure of the first buffer and the second buffer provided in an embodiment of the present application. The first buffer includes a first sub-region and a second sub-region, and the second buffer includes a third sub-region and a fourth sub-region. Either the first sub-region or the second sub-region can be used to store the first sub-matrix. Either the third sub-region or the fourth sub-region can be used to store the second sub-matrix.

[0198] In some embodiments, transferring the candidate first sub-matrix from a preset buffer area to a first buffer area of ​​a computing unit corresponding to each sub-matrix to be updated includes:

[0199] (1.2.1) determining a first idle sub-area in the first sub-area and the second sub-area;

[0200] (1.2.2) Moving the candidate first sub-matrix from the preset buffer area to the first free sub-region.

[0201] If there is no matrix data in either the first sub-region or the second sub-region, the sub-region may be determined as a first idle sub-region, and then the candidate first sub-matrix is ​​moved from the preset buffer area to the first idle sub-region.

[0202] When the matrix calculation unit reads the target first submatrix that requires general matrix multiplication operation corresponding to the submatrix to be updated from the first subregion or the second subregion, the occupied subregion corresponding to the target first submatrix cannot transmit the candidate first submatrix.

[0203] After the matrix calculation unit reads and calculates the target first submatrix, it can delete data from the occupied subregion corresponding to the target first submatrix. For example, if the occupied subregion is the first subregion, the target first submatrix can be deleted from the first subregion and the first subregion is updated to the first free subregion for storing other candidate first submatrices. The matrix calculation unit can then read and use the candidate first submatrix in the second subregion.

[0204] In some embodiments, transferring the candidate second sub-matrix to the second buffer of the computing unit corresponding to each sub-matrix to be updated includes:

[0205] (1.4.1) Determine a second idle sub-area in the third sub-area and the fourth sub-area;

[0206] (1.4.2) Moving the candidate second sub-matrix from the preset buffer area to the second free sub-region.

[0207] Among them, in the third sub-region and the fourth sub-region, if any sub-region has no matrix data, the sub-region can be determined as the second idle sub-region, and then the candidate second sub-matrix is ​​moved from the preset buffer area to the second idle sub-region.

[0208] When the matrix calculation unit reads the target second submatrix requiring general matrix multiplication operation corresponding to the submatrix to be updated from the third subregion or the fourth subregion, the occupied subregion corresponding to the target second submatrix cannot transmit the candidate second submatrix.

[0209] After the matrix calculation unit reads and calculates the target second submatrix, it can delete data from the occupied subregion corresponding to the target second submatrix. For example, if the occupied subregion is the third subregion, the target second submatrix can be deleted from the third subregion and the third subregion can be updated to the second free subregion for storing other candidate second submatrices. The matrix calculation unit can then read and use the candidate second submatrix in the fourth subregion.

[0210] As can be seen from the above, in this application, dual storage areas are set for the first buffer and the second buffer of the computing unit, so that the first buffer and the second buffer can continuously update and store the sub-matrices to be calculated subsequently, thereby improving the computing efficiency of the computing unit.

[0211] In step 150, a general matrix multiplication operation is performed on each submatrix to be updated, a target first submatrix corresponding to each submatrix to be updated, and a target second submatrix according to the calculation unit corresponding to each submatrix to be updated to obtain an update submatrix corresponding to each submatrix to be updated, and an update matrix corresponding to the matrix to be updated is generated according to each update submatrix.

[0212] In the present application, the calculation formula for the general matrix performed by the calculation unit is: C←αAB+βC. Where A is the first matrix, B is the second matrix, C on the right side of the formula is the matrix to be updated, C on the left side of the formula is the update matrix, α is the first preset floating point number, and β is the second preset floating point number.

[0213] Therefore, when updating a submatrix to be updated, the first and second submatrices corresponding to the submatrix to be updated can be first determined. The first preset floating-point number, the first submatrix, and the second submatrix are then multiplied together to obtain a first multiplication result, which can be understood as the target submatrix. The second preset floating-point number is then multiplied by the submatrix to be updated to obtain a second multiplication result. Finally, the first and second multiplication results are added together to obtain an updated submatrix corresponding to the submatrix to be updated. An update matrix corresponding to the matrix to be updated can be generated from each update submatrix. For example, each update submatrix is ​​updated to the matrix row and column position of the corresponding matrix to be updated in the matrix to be updated, thereby generating an update matrix corresponding to the matrix to be updated.

[0214] In some embodiments, performing a general matrix multiplication operation on each submatrix to be updated, a target first submatrix corresponding to each submatrix to be updated, and a target second submatrix according to a calculation unit corresponding to each submatrix to be updated to obtain an updated submatrix corresponding to each submatrix to be updated includes:

[0215] (1.1) Inputting the target first submatrix and the target second submatrix into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated to perform matrix multiplication operation to obtain the target submatrix corresponding to each submatrix to be updated;

[0216] (1.2) Each sub-matrix to be updated and the target sub-matrix are input into the vector calculation unit of the calculation unit corresponding to each sub-matrix to be updated, and a vector addition operation is performed to obtain an updated sub-matrix corresponding to each sub-matrix to be updated.

[0217] From the above content, it can be seen that the calculation unit includes a matrix calculation unit and a vector calculation unit. The target first sub-matrix and the target second sub-matrix can be input into the matrix calculation unit corresponding to each sub-matrix to be updated to perform matrix multiplication operations to obtain the target sub-matrix corresponding to each sub-matrix to be updated.

[0218] Then, each sub-matrix to be updated and the target sub-matrix are input into the vector calculation unit of the calculation unit corresponding to each sub-matrix to be updated to perform vector addition operation, so as to obtain the updated sub-matrix corresponding to each sub-matrix to be updated.

[0219] The matrix calculation unit is more suitable for matrix multiplication calculations, while the vector calculation unit is more suitable for vector addition operations. By assigning different calculation steps to different units for calculation, the calculation efficiency of the calculation unit can be improved.

[0220] In some embodiments, after inputting the target first submatrix and the target second submatrix into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated to perform a matrix multiplication operation to obtain the target submatrix corresponding to each submatrix to be updated, the method further includes:

[0221] (2.1) deleting the target first sub-matrix from the first buffer to update the first free sub-region in the first buffer;

[0222] (2.2) The target second sub-matrix is ​​deleted from the second buffer to update the second free sub-region in the second buffer.

[0223] It can be understood that after the target first submatrix and the target second submatrix are input into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated for matrix multiplication operation, and the target submatrix corresponding to each submatrix to be updated is obtained, the matrix calculation unit has completed the calculation of the target first submatrix and the target second submatrix. Therefore, the target first submatrix can be deleted from the first buffer to update the first free sub-region in the first buffer. For example, if the target first submatrix occupies the first sub-region, it can be deleted from the first sub-region, thereby updating the first sub-region to the first free sub-region. For example, if the target second submatrix occupies the third sub-region, it can be deleted from the third sub-region, thereby updating the third sub-region to the second free sub-region.

[0224] The advantage of this approach is that the free sub-regions in the first and second buffers can be continuously updated, so that the candidate first sub-matrix that needs to be subsequently multiplied is input into the first buffer, and the candidate second sub-matrix that needs to be subsequently multiplied is input into the second buffer. This improves the data transmission efficiency of the computing unit and thus improves the computing efficiency of the computing unit.

[0225] It should be noted that in the present application, the matrix to be updated, the first matrix and the second matrix are divided into blocks of reasonable sizes, so that the first buffer of each computing unit can store the first sub-matrix that meets its first threshold size, and the second buffer of each computing unit can store the second sub-matrix that meets its second threshold size. This can improve the data transmission and storage efficiency, so that the matrix calculation unit can process different sub-matrices in a program to improve the computational efficiency of the entire computing unit.

[0226] Secondly, by evenly distributing multiple matrices to be updated to each computing unit, load balancing of the computing units can be achieved without wasting the computing power of the computing units, thereby improving the computing efficiency of the computing units.

[0227] Furthermore, before each computing unit performs a general matrix multiplication operation, the first sub-matrix can be pre-stored in a preset cache area and the second sub-matrix can be pre-stored in a transfer cache area to reduce the number of interactions between the computing unit and the global memory, thereby improving the computing efficiency of the computing unit.

[0228] In addition, by providing dual storage areas in the first buffer and the second buffer of the computing unit, the matrix data to be calculated can be continuously transferred to the first buffer and the second buffer, thereby improving the computing efficiency of the computing unit.

[0229] In this application, the above-mentioned method is used to improve the computing efficiency of each computing unit, that is, the computing efficiency of the computing unit when performing general matrix multiplication operations is improved, thereby improving the computing efficiency of the neural network processor when performing general matrix multiplication operations, and effectively exerting the computing power of the neural network processor.

[0230] As can be seen from the above, in an embodiment of the present application, the neural network processor includes a global memory, a transfer buffer area and multiple computing units, and obtains multiple sub-matrices to be updated by block processing of the matrix to be updated, and the multiple sub-matrices to be updated are evenly distributed to each computing unit; a first sub-matrix corresponding to each sub-matrix to be updated is determined in the first matrix, and the first sub-matrix is ​​moved from the global memory to the preset buffer area of ​​the computing unit corresponding to each sub-matrix to be updated; a second sub-matrix corresponding to each sub-matrix to be updated is determined in the second matrix, and the second sub-matrix is ​​moved from the global memory to the transfer buffer area; a target first sub-matrix requiring a general matrix multiplication operation corresponding to each sub-matrix to be updated is obtained from the preset buffer area, and a target second sub-matrix requiring a general matrix multiplication operation is obtained from the transfer buffer area for each sub-matrix to be updated; a general matrix multiplication operation is performed on each sub-matrix to be updated, a target first sub-matrix corresponding to each sub-matrix to be updated and a target second sub-matrix corresponding to each sub-matrix to be updated according to the computing unit corresponding to each sub-matrix to be updated, to obtain an updated sub-matrix corresponding to each sub-matrix to be updated, and an updated matrix corresponding to the matrix to be updated is generated according to each updated sub-matrix.

[0231] In this way, by performing block processing on the matrix to be updated, multiple sub-matrices to be updated are obtained, and the first matrix and the second matrix are processed in blocks to obtain the first sub-matrix and the second sub-matrix corresponding to each sub-matrix to be updated, and then the first sub-matrix and the second sub-matrix that each computing unit needs to process are moved from the global memory to the corresponding cache area in advance. When each computing unit performs a general matrix multiplication operation subsequently, the number of times data is moved from the global memory can be reduced, and each computing unit processes the evenly distributed sub-matrix to be updated, the first sub-matrix and the second sub-matrix, which can achieve load balancing when multiple computing units perform general matrix multiplication operations, thereby improving the efficiency of each computing unit when performing general matrix multiplication operations, thereby efficiently exerting the computing power of the neural network processor.

[0232] See also Figure 9 , Figure 9 : is another flow chart of the data processing method provided in an embodiment of the present application. The data processing method may include the following steps:

[0233] Step 201: Divide the matrix to be updated into blocks to obtain multiple sub-matrices to be updated, and evenly distribute the multiple sub-matrices to be updated to each computing unit;

[0234] Step 202: Determine a first threshold size of a matrix that can be stored in a first buffer of each computing unit, the first buffer being used to obtain and store a target first submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from a preset cache area;

[0235] Step 203: Divide the first matrix into blocks according to a first threshold size to obtain a plurality of first sub-matrices;

[0236] Step 204: Determine a first sub-matrix corresponding to each sub-matrix to be updated from the plurality of first sub-matrices;

[0237] Step 205: Determine a second threshold size of a matrix that can be stored in the second buffer of each computing unit, the second buffer being used to obtain and store a target second submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from the transfer buffer area;

[0238] Step 206: Divide the second matrix into blocks according to the second threshold size to obtain multiple second sub-matrices;

[0239] Step 207: Determine, from the plurality of second sub-matrices, a second sub-matrix corresponding to each sub-matrix to be updated;

[0240] Step 208: Determine in a preset buffer area a candidate first sub-matrix corresponding to each sub-matrix to be updated;

[0241] Step 209: Move the candidate first submatrices from the preset buffer area to the first buffer area of ​​the calculation unit corresponding to each submatrix to be updated, and obtain the target first submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated from the candidate first submatrices in the first buffer area;

[0242] Step 210: Determine in the transfer buffer area a candidate second sub-matrix corresponding to each sub-matrix to be updated;

[0243] Step 211: Move the candidate second submatrices to the second buffer of the calculation unit corresponding to each submatrix to be updated, and obtain the target second submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated from the candidate second submatrices in the second buffer;

[0244] Step 212: Input the target first submatrix and the target second submatrix into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated to perform a matrix multiplication operation, thereby obtaining a target submatrix corresponding to each submatrix to be updated;

[0245] Step 213: Input each sub-matrix to be updated and the target sub-matrix into the vector calculation unit of the calculation unit corresponding to each sub-matrix to be updated to perform vector addition operation to obtain an update sub-matrix corresponding to each sub-matrix to be updated, and generate an update matrix corresponding to the matrix to be updated based on each update sub-matrix.

[0246] In the above embodiments, the description of each embodiment has its own focus. For the part that is not described in detail in a certain embodiment, please refer to the detailed description of the data processing method above, and will not be repeated here.

[0247] See also Figure 10 , Figure 10 This application implements a structural diagram of a data processing device that can execute the data processing method described in the above embodiment.

[0248] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0249] like Figure 10 As shown, the data processing device 300 is applied to a neural network processor, which includes a global memory, a transfer buffer area, and multiple computing units. The data processing device 300 includes:

[0250] A block division module 310 is configured to divide the matrix to be updated into blocks to obtain multiple sub-matrices to be updated, and evenly distribute the multiple sub-matrices to be updated to each computing unit;

[0251] A first determining module 320 is configured to determine a first submatrix corresponding to each submatrix to be updated in the first matrix, and move the first submatrix from the global memory to a preset buffer area of ​​a computing unit corresponding to each submatrix to be updated;

[0252] A second determining module 330 is configured to determine, in the second matrix, a second submatrix corresponding to each submatrix to be updated, and transfer the second submatrix from the global memory to the transfer buffer;

[0253] An acquisition module 340 is configured to acquire a target first submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from a preset buffer, and acquire a target second submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from a transfer buffer;

[0254] The computing module 350 is configured to perform a general matrix multiplication operation on each submatrix to be updated, a target first submatrix corresponding to each submatrix to be updated, and a target second submatrix according to the computing unit corresponding to each submatrix to be updated, to obtain an update submatrix corresponding to each submatrix to be updated, and to generate an update matrix corresponding to the matrix to be updated according to each update submatrix.

[0255] In some implementations, the first determining module 320 is configured to:

[0256] Obtain a first matrix, and perform block processing on the first matrix to obtain a first target matrix corresponding to each submatrix to be updated;

[0257] Performing grouping processing on the first target matrix to obtain a first sub-matrix corresponding to each sub-matrix to be updated;

[0258] Each first sub-matrix is ​​moved from the global memory to a preset buffer area in the corresponding computing unit.

[0259] In some embodiments, the first determination module 320 includes a first determination submodule, a blocking submodule, and a second determination submodule;

[0260] A first determining submodule is configured to determine a first threshold size of a matrix that can be stored in a first buffer of each computing unit, the first buffer being configured to obtain and store a target first submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from a preset cache area;

[0261] A block submodule, configured to perform block processing on the first matrix according to a first threshold size to obtain a plurality of first sub-matrices;

[0262] The second determining submodule is configured to determine, from the plurality of first sub-matrices, a first sub-matrix corresponding to each sub-matrix to be updated.

[0263] In some embodiments, the second determining submodule is configured to:

[0264] Determining a first threshold size of a matrix that can be stored in a first buffer of each computing unit, the first buffer being used to obtain and store a target first submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from a preset cache area;

[0265] Divide the first matrix into blocks according to a first threshold size to obtain a plurality of first sub-matrices;

[0266] A first sub-matrix corresponding to each sub-matrix to be updated is determined from the plurality of first sub-matrices.

[0267] In some implementations, the first determining module 320 is configured to:

[0268] Determine the capacity of a preset buffer area of ​​a computing unit corresponding to each sub-matrix to be updated;

[0269] According to the capacity and the first row and column information, determining the first sub-matrix that needs to be moved currently in the first sub-matrix that matches the rows and columns of each sub-matrix to be updated;

[0270] The first sub-matrix that needs to be moved currently is moved from the global memory to the preset buffer area of ​​the calculation unit corresponding to each sub-matrix to be updated.

[0271] In some implementations, the second determining module 330 is configured to:

[0272] Determining a second threshold size of a matrix that can be stored in a second buffer of each computing unit, the second buffer being used to obtain and store a target second submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from the transfer buffer area;

[0273] Divide the second matrix into blocks according to a second threshold size to obtain a plurality of second sub-matrices;

[0274] A second sub-matrix corresponding to each sub-matrix to be updated is determined from the plurality of second sub-matrices.

[0275] In some embodiments, the acquisition module 340 is configured to:

[0276] Determining a candidate first sub-matrix corresponding to each sub-matrix to be updated in a preset buffer area;

[0277] Transferring the candidate first submatrices from the preset buffer area to the first buffer area of ​​the calculation unit corresponding to each submatrix to be updated, and obtaining the target first submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated from the candidate first submatrix in the first buffer area;

[0278] Determining, in the transfer buffer area, a candidate second submatrix corresponding to each submatrix to be updated;

[0279] The candidate second submatrix is ​​transferred to the second buffer of the calculation unit corresponding to each submatrix to be updated, and the target second submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated is obtained from the candidate second submatrix in the second buffer.

[0280] In some embodiments, the first buffer area includes a first sub-area and a second sub-area, the second buffer area includes a third sub-area and a fourth sub-area, and the first determining module 320 is configured to:

[0281] Determine a first idle sub-area in the first sub-area and the second sub-area;

[0282] Moving the candidate first sub-matrix from the preset buffer area to the first idle sub-area;

[0283] The second determining module 330 is configured to:

[0284] determining a second idle sub-area in the third sub-area and the fourth sub-area;

[0285] The candidate second sub-matrix is ​​moved from the preset buffer area to the second idle sub-region.

[0286] In some embodiments, the computing module 350 is configured to:

[0287] Inputting the target first submatrix and the target second submatrix into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated to perform matrix multiplication operation, thereby obtaining the target submatrix corresponding to each submatrix to be updated;

[0288] Each sub-matrix to be updated and the target sub-matrix are input into the vector calculation unit of the calculation unit corresponding to each sub-matrix to be updated to perform vector addition operation, so as to obtain the updated sub-matrix corresponding to each sub-matrix to be updated.

[0289] In some embodiments, after inputting the target first submatrix and the target second submatrix into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated to perform a matrix multiplication operation to obtain the target submatrix corresponding to each submatrix to be updated, the calculation module 350 is configured to:

[0290] Deleting the target first sub-matrix from the first buffer to update a first idle sub-region in the first buffer;

[0291] The target second sub-matrix is ​​deleted from the second buffer to update a second idle sub-region in the second buffer.

[0292] In some embodiments, the block module 310 is configured to:

[0293] determining a total number of the plurality of computing units;

[0294] Multiple matrices to be updated are evenly distributed to each computing unit according to the total number.

[0295] In the above embodiments, the description of each embodiment has its own focus. For the part that is not described in detail in a certain embodiment, please refer to the detailed description of the data processing method above, and will not be repeated here.

[0296] In the embodiment of the present application, the neural network processor includes a global memory, a transfer buffer area and multiple computing units, a block module 310, which is used to block the matrix to be updated to obtain multiple sub-matrices to be updated, and evenly distribute the multiple sub-matrices to be updated to each computing unit; a first determination module 320, which is used to determine the first sub-matrix corresponding to each sub-matrix to be updated in the first matrix, and move the first sub-matrix from the global memory to the preset buffer area of ​​the computing unit corresponding to each sub-matrix to be updated; a second determination module 330, which is used to determine the second sub-matrix corresponding to each sub-matrix to be updated in the second matrix, and move the second sub-matrix from the global memory to the preset buffer area of ​​the computing unit corresponding to each sub-matrix to be updated; The matrix is ​​transferred from the storage to the transfer buffer area; an acquisition module 340 is used to obtain a target first submatrix that requires a general matrix multiplication operation corresponding to each submatrix to be updated from the preset buffer area, and obtain a target second submatrix that requires a general matrix multiplication operation corresponding to each submatrix to be updated from the transfer buffer area; a calculation module 350 is used to perform a general matrix multiplication operation on each submatrix to be updated, the target first submatrix corresponding to each submatrix to be updated, and the target second submatrix corresponding to each submatrix to be updated according to the calculation unit corresponding to each submatrix to be updated, to obtain an updated submatrix corresponding to each submatrix to be updated, and to generate an update matrix corresponding to the matrix to be updated according to each updated submatrix.

[0297] In this way, by performing block processing on the matrix to be updated, multiple sub-matrices to be updated are obtained, and the first matrix and the second matrix are processed in blocks to obtain the first sub-matrix and the second sub-matrix corresponding to each sub-matrix to be updated, and then the first sub-matrix and the second sub-matrix that each computing unit needs to process are moved from the global memory to the corresponding cache area in advance. When each computing unit performs a general matrix multiplication operation subsequently, the number of times data is moved from the global memory can be reduced, and each computing unit processes the evenly distributed sub-matrix to be updated, the first sub-matrix and the second sub-matrix, which can achieve load balancing when multiple computing units perform general matrix multiplication operations, thereby improving the efficiency of each computing unit when performing general matrix multiplication operations, thereby efficiently exerting the computing power of the neural network processor.

[0298] See also Figure 11 , Figure 11 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. This embodiment of the present application also provides a computer device comprising a memory and a processor. The memory stores a computer program, and the processor implements the aforementioned data processing method when executing the computer program. The computer device can be any intelligent terminal, including a tablet computer and an in-vehicle computer.

[0299] See also Figure 11 , Figure 11 The hardware structure of a computer device according to another embodiment is shown. The computer device includes:

[0300] Processor 401, which may specifically be a neural network processor, is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0301] The memory 402 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 402 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 402 and is called by the processor 401 to execute the data processing method of the embodiments of this application;

[0302] Input / output interface 403, used to implement information input and output;

[0303] Communication interface 404, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0304] Bus 405 , which transmits information between various components of the device (e.g., processor 401 , memory 402 , input / output interface 403 , and communication interface 404 );

[0305] The processor 401 , the memory 402 , the input / output interface 403 and the communication interface 404 are connected to each other in communication within the device via a bus 405 .

[0306] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned data processing method is implemented.

[0307] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0308] The data processing method, apparatus, computer equipment, and storage medium provided in the embodiments of the present application obtain multiple sub-matrices to be updated by performing block processing on a matrix to be updated, and evenly distribute the multiple sub-matrices to be updated to each computing unit; determine a first sub-matrix corresponding to each sub-matrix to be updated in a first matrix, and move the first sub-matrix from a global memory to a preset cache area of ​​a computing unit corresponding to each sub-matrix to be updated; determine a second sub-matrix corresponding to each sub-matrix to be updated in a second matrix, and move the second sub-matrix from the global memory to a transfer cache area; obtain a target first sub-matrix requiring a general matrix multiplication operation corresponding to each sub-matrix to be updated from the preset cache area, and obtain a target second sub-matrix requiring a general matrix multiplication operation corresponding to each sub-matrix to be updated from the transfer cache area; perform a general matrix multiplication operation on each sub-matrix to be updated, a target first sub-matrix corresponding to each sub-matrix to be updated, and a target second sub-matrix corresponding to each sub-matrix to be updated according to the computing unit corresponding to each sub-matrix to be updated, to obtain an updated sub-matrix corresponding to each sub-matrix to be updated, and generate an updated matrix corresponding to the matrix to be updated based on each updated sub-matrix.

[0309] In this way, by performing block processing on the matrix to be updated, multiple sub-matrices to be updated are obtained, and the first matrix and the second matrix are processed in blocks to obtain the first sub-matrix and the second sub-matrix corresponding to each sub-matrix to be updated, and then the first sub-matrix and the second sub-matrix that each computing unit needs to process are moved from the global memory to the corresponding cache area in advance. When each computing unit performs a general matrix multiplication operation subsequently, the number of times data is moved from the global memory can be reduced, and each computing unit processes the evenly distributed sub-matrix to be updated, the first sub-matrix and the second sub-matrix, which can achieve load balancing when multiple computing units perform general matrix multiplication operations, thereby improving the efficiency of each computing unit when performing general matrix multiplication operations, thereby efficiently exerting the computing power of the neural network processor.

[0310] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0311] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0312] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0313] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0314] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0315] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0316] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0317] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0318] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0319] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0320] The above is a detailed introduction to a data processing method, device, computer equipment and storage medium provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. At the same time, for those skilled in the art, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes, but this does not limit the scope of rights of the embodiments of the present application. Any modifications, equivalent substitutions and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application should be within the scope of rights of the embodiments of the present application.

Claims

1. A data processing method, characterized in that: Applied to a neural network processor, the neural network processor includes a global memory, a transfer buffer area and multiple computing units, the method includes: Performing block processing on the matrix to be updated to obtain multiple sub-matrices to be updated, and evenly distributing the multiple sub-matrices to be updated to each computing unit; Determining a first submatrix corresponding to each submatrix to be updated in the first matrix, and moving the first submatrix from the global memory to a preset cache area of ​​a computing unit corresponding to each submatrix to be updated; Determining, in the second matrix, a second submatrix corresponding to each of the submatrices to be updated, and transferring the second submatrix from the global memory to the transfer buffer area; Determining, in the preset buffer area, a candidate first sub-matrix corresponding to each sub-matrix to be updated; Transferring the candidate first submatrix from the preset buffer area to a first buffer area of ​​a calculation unit corresponding to each submatrix to be updated, and obtaining a target first submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from the candidate first submatrix in the first buffer area; Determining, in the transfer buffer area, a candidate second submatrix corresponding to each submatrix to be updated; Transferring the candidate second submatrix to a second buffer of a calculation unit corresponding to each submatrix to be updated, and obtaining, from the candidate second submatrix in the second buffer, a target second submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated; A general matrix multiplication operation is performed on each submatrix to be updated, the target first submatrix corresponding to each submatrix to be updated, and the target second submatrix according to the calculation unit corresponding to each submatrix to be updated to obtain an update submatrix corresponding to each submatrix to be updated, and an update matrix corresponding to the matrix to be updated is generated according to each update submatrix.

2. The data processing method according to claim 1, wherein: The step of determining in the first matrix a first submatrix corresponding to each submatrix to be updated includes: Determining a first threshold size of a matrix that can be stored in a first buffer of each computing unit, the first buffer being used to obtain and store a target first submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from the preset cache area; Performing block processing on the first matrix according to the first threshold size to obtain a plurality of first sub-matrices; A first sub-matrix corresponding to each of the sub-matrices to be updated is determined from the multiple first sub-matrices.

3. The data processing method according to claim 2, characterized in that: The determining, from the plurality of first sub-matrices, a first sub-matrix corresponding to each sub-matrix to be updated includes: Determining target row and column information of each to-be-updated submatrix in the to-be-updated matrix, and determining first row and column information of each first submatrix in the plurality of first submatrices in the first matrix; According to the first row and column information, a first sub-matrix that matches the target row and column information of each sub-matrix to be updated is determined from the multiple first sub-matrices.

4. The data processing method according to claim 3, wherein: The moving the first sub-matrix from the global memory to a preset cache area of ​​a computing unit corresponding to each sub-matrix to be updated includes: Determining the capacity of a preset buffer area of ​​a computing unit corresponding to each sub-matrix to be updated; Determining, according to the capacity and the first row and column information, a first sub-matrix that needs to be moved currently in the first sub-matrices that match the rows and columns of each sub-matrix to be updated; The first sub-matrix that currently needs to be transferred is transferred from the global memory to a preset buffer area of ​​the calculation unit corresponding to each sub-matrix to be updated.

5. The data processing method according to claim 1, wherein: The determining, in the second matrix, a second submatrix corresponding to each submatrix to be updated includes: Determining a second threshold size of a matrix that can be stored in a second buffer of each computing unit, wherein the second buffer is used to obtain and store a target second sub-matrix requiring a general matrix multiplication operation corresponding to each sub-matrix to be updated from the transfer buffer area; performing block processing on the second matrix according to the second threshold size to obtain a plurality of second sub-matrices; A second sub-matrix corresponding to each of the sub-matrices to be updated is determined from the multiple second sub-matrices.

6. The data processing method according to claim 1, wherein: The first buffer includes a first sub-region and a second sub-region, the second buffer includes a third sub-region and a fourth sub-region, and the moving the candidate first sub-matrix from the preset cache area to the first buffer of the calculation unit corresponding to each sub-matrix to be updated includes: Determine a first idle sub-area in the first sub-area and the second sub-area; Moving the candidate first sub-matrix from the preset buffer area to the first idle sub-area; The step of transferring the candidate second sub-matrix to the second buffer of the computing unit corresponding to each sub-matrix to be updated includes: determining a second idle sub-area in the third sub-area and the fourth sub-area; The candidate second sub-matrix is ​​moved from the preset buffer area to the second idle sub-region.

7. The data processing method according to claim 1, wherein: The performing a general matrix multiplication operation on each submatrix to be updated, the target first submatrix corresponding to each submatrix to be updated, and the target second submatrix according to the calculation unit corresponding to each submatrix to be updated, to obtain an updated submatrix corresponding to each submatrix to be updated, includes: Inputting the target first submatrix and the target second submatrix into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated to perform a matrix multiplication operation to obtain a target submatrix corresponding to each submatrix to be updated; Each of the sub-matrices to be updated and the target sub-matrix is ​​input into a vector calculation unit of a calculation unit corresponding to each of the sub-matrices to be updated, and a vector addition operation is performed to obtain an updated sub-matrix corresponding to each of the sub-matrices to be updated.

8. The data processing method according to claim 7, characterized in that: After inputting the target first submatrix and the target second submatrix into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated to perform a matrix multiplication operation to obtain the target submatrix corresponding to each submatrix to be updated, the method further includes: Deleting the target first sub-matrix from the first buffer to update a first idle sub-region in the first buffer; The target second sub-matrix is ​​deleted from the second buffer to update a second idle sub-region in the second buffer.

9. The data processing method according to claim 1, wherein: The step of evenly distributing the plurality of sub-matrices to be updated to each computing unit comprises: determining a total number of the plurality of computing units; The multiple sub-matrices to be updated are evenly distributed to each computing unit according to the total number.

10. A data processing device, characterized in that: Applied to a neural network processor, the neural network processor includes a global memory, a transfer buffer area and multiple computing units, the device includes: A block module, configured to perform block processing on the matrix to be updated to obtain multiple sub-matrices to be updated, and evenly distribute the multiple sub-matrices to be updated to each computing unit; a first determining module, configured to determine, in the first matrix, a first submatrix corresponding to each submatrix to be updated, and transfer the first submatrix from the global memory to a preset buffer area of ​​a computing unit corresponding to each submatrix to be updated; A second determining module is configured to determine, in the second matrix, a second submatrix corresponding to each of the submatrices to be updated, and transfer the second submatrix from the global memory to the transfer buffer area; An acquisition module, configured to determine, in the preset buffer area, a candidate first sub-matrix corresponding to each sub-matrix to be updated; Transferring the candidate first submatrix from the preset buffer area to a first buffer area of ​​a calculation unit corresponding to each submatrix to be updated, and obtaining a target first submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from the candidate first submatrix in the first buffer area; Determining, in the transfer buffer area, a candidate second submatrix corresponding to each submatrix to be updated; Transferring the candidate second submatrix to a second buffer of a calculation unit corresponding to each submatrix to be updated, and obtaining, from the candidate second submatrix in the second buffer, a target second submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated; a calculation module, configured to perform a general matrix multiplication operation on each submatrix to be updated, the target first submatrix corresponding to each submatrix to be updated, and the target second submatrix according to the calculation unit corresponding to each submatrix to be updated, to obtain an update submatrix corresponding to each submatrix to be updated, and to generate an update matrix corresponding to the matrix to be updated according to each update submatrix.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the data processing method according to any one of claims 1 to 9.

12. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the data processing method according to any one of claims 1 to 9 is implemented.

Images