Data processing method and device, computer equipment and storage medium

Abstract

The embodiment of the invention discloses a data processing method and device, computer equipment and a storage medium, a neural network processor comprises a global memory, a transfer cache region and a plurality of calculation units, a to-be-updated matrix is subjected to block processing to obtain a plurality of to-be-updated sub-matrixes, and the plurality of to-be-updated sub-matrixes are evenly distributed to each calculation unit; determining a first sub-matrix corresponding to each to-be-updated sub-matrix in the first matrix, and carrying the first sub-matrix from the global memory to a preset cache region of a calculation unit corresponding to each to-be-updated sub-matrix; determining a second sub-matrix corresponding to each to-be-updated sub-matrix in the second matrix, and carrying the second sub-matrixes from the global memory to a transfer cache region; obtaining a target first sub-matrix from the preset cache region, and obtaining a target second sub-matrix from the transfer cache region; and performing general matrix multiplication operation on each to-be-updated sub-matrix, the target first sub-matrix and the target second sub-matrix through a calculation unit to obtain an updated sub-matrix.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, apparatus, computer equipment and storage medium. Background Art

[0002] With the development of artificial intelligence technology, artificial intelligence-related pending data currently needs to be processed by a specific processor to perform data operations on the pending data. The most commonly used processor is the graphics processing unit (GPU), which can perform matrix operations, vector operations, etc. For example, the graphics processor can perform general matrix multiplication (Gemm, General Matrix Multiplication) operations.

[0003] In the related art, when a graphics processor performs general matrix multiplication operations on a matrix, a relatively efficient algorithm can be used to perform the operation. However, when a neural network processor (NPU) performs general matrix multiplication operations, due to the difference in the computing units of the graphics processor and the neural network processor, the general matrix multiplication-related algorithms applicable to the graphics processor are not applicable to the neural network processor. Although some neural network processors have their own general matrix multiplication algorithms, the general matrix multiplication algorithms they come with cannot efficiently bring into play the computing power of the neural network processor, resulting in a waste of computing power of the neural network processor.

[0004] Therefore, when the neural network processor in the related art performs general matrix multiplication operations, the operation efficiency is low, resulting in the inability to efficiently exert the computing power of the neural network processor. Summary of the invention

[0005] The embodiments of the present application provide a data processing method, apparatus, computer device and storage medium, which can improve the computing efficiency of a neural network processor when performing general matrix multiplication operations, thereby effectively exerting the computing power of the neural network processor.

[0006] In order to achieve the above-mentioned object, a first aspect of an embodiment of the present application provides a data processing method, which is applied to a neural network processor, wherein the neural network processor includes a global memory, a transfer buffer area, and a plurality of computing units, and the method includes:

[0007] Performing block processing on the matrix to be updated to obtain multiple sub-matrices to be updated, and evenly distributing the multiple sub-matrices to be updated to each computing unit;

[0008] Determine in the first matrix a first sub-matrix corresponding to each sub-matrix to be updated, and move the first sub-matrix from the global memory to a preset cache area of ​​a computing unit corresponding to each sub-matrix to be updated;

[0009] Determine in the second matrix a second sub-matrix corresponding to each of the sub-matrices to be updated, and move the second sub-matrix from the global memory to the transfer buffer area;

[0010] Obtaining a target first submatrix that requires a general matrix multiplication operation corresponding to each submatrix to be updated from the preset buffer area, and obtaining a target second submatrix that requires a general matrix multiplication operation corresponding to each submatrix to be updated from the transfer buffer area;

[0011] According to the calculation unit corresponding to each sub-matrix to be updated, a general matrix multiplication operation is performed on each sub-matrix to be updated, the target first sub-matrix corresponding to each sub-matrix to be updated, and the target second sub-matrix to obtain an update sub-matrix corresponding to each sub-matrix to be updated, and an update matrix corresponding to the matrix to be updated is generated according to each update sub-matrix.

[0012] In order to achieve the above-mentioned object, a second aspect of an embodiment of the present application provides a data processing device, which is applied to a neural network processor, wherein the neural network processor includes a global memory, a transfer buffer area, and a plurality of computing units, and the device includes:

[0013] A block division module, used for performing block processing on the matrix to be updated to obtain multiple sub-matrices to be updated, and evenly distributing the multiple sub-matrices to be updated to each computing unit;

[0014] A first determination module, configured to determine a first sub-matrix corresponding to each sub-matrix to be updated in the first matrix, and move the first sub-matrix from the global memory to a preset cache area of ​​a calculation unit corresponding to each sub-matrix to be updated;

[0015] A second determination module, configured to determine in the second matrix a second sub-matrix corresponding to each sub-matrix to be updated, and to transfer the second sub-matrix from the global memory to the transfer buffer area;

[0016] An acquisition module, configured to acquire a target first submatrix that requires a general matrix multiplication operation and corresponds to each submatrix to be updated from the preset buffer area, and acquire a target second submatrix that requires a general matrix multiplication operation and corresponds to each submatrix to be updated from the transfer buffer area;

[0017] A calculation module is used to perform a general matrix multiplication operation on each sub-matrix to be updated, the target first sub-matrix corresponding to each sub-matrix to be updated, and the target second sub-matrix according to the calculation unit corresponding to each sub-matrix to be updated, to obtain an update sub-matrix corresponding to each sub-matrix to be updated, and to generate an update matrix corresponding to the matrix to be updated according to each update sub-matrix.

[0018] In some implementations, the first determining module is configured to:

[0019] Obtain a first matrix, and perform block processing on the first matrix to obtain a first target matrix corresponding to each sub-matrix to be updated;

[0020] Performing group processing on the first target matrix to obtain a first sub-matrix corresponding to each sub-matrix to be updated;

[0021] Each first sub-matrix is ​​transferred from the global memory to a preset buffer area in the corresponding computing unit.

[0022] In some embodiments, the first determination module includes a first determination submodule, a block division submodule, and a second determination submodule;

[0023] A first determination submodule, used to determine a first threshold size of a matrix that can be stored in a first buffer of each computing unit, the first buffer being used to obtain and store a target first submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from the preset cache area;

[0024] A block submodule, configured to perform block processing on the first matrix according to the first threshold size to obtain a plurality of first sub-matrices;

[0025] The second determining submodule is used to determine, from the multiple first sub-matrices, a first sub-matrix corresponding to each of the sub-matrices to be updated.

[0026] In some implementations, the second determining submodule is configured to:

[0027] Determine a first threshold size of a matrix that can be stored in a first buffer of each computing unit, the first buffer being used to obtain and store a target first submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from the preset cache area;

[0028] Performing block processing on the first matrix according to the first threshold size to obtain a plurality of first sub-matrices;

[0029] A first sub-matrix corresponding to each sub-matrix to be updated is determined from the multiple first sub-matrices.

[0030] In some implementations, the first determining module is configured to:

[0031] Determining the capacity of a preset buffer area of ​​a computing unit corresponding to each sub-matrix to be updated;

[0032] According to the capacity and the first row and column information, determining a first sub-matrix that needs to be moved currently in the first sub-matrixes that match the rows and columns of each sub-matrix to be updated;

[0033] The first sub-matrix currently required to be transferred is transferred from the global memory to a preset buffer area of ​​the calculation unit corresponding to each sub-matrix to be updated.

[0034] In some implementations, the second determining module is configured to:

[0035] Determining a second threshold size of a matrix that can be stored in a second buffer of each computing unit, wherein the second buffer is used to obtain and store a target second sub-matrix that requires a general matrix multiplication operation and corresponds to each sub-matrix to be updated from the transfer buffer area;

[0036] Performing block processing on the second matrix according to the second threshold size to obtain a plurality of second sub-matrices;

[0037] A second sub-matrix corresponding to each of the sub-matrices to be updated is determined from the multiple second sub-matrices.

[0038] In some embodiments, the acquisition module is used to:

[0039] Determining in the preset buffer area a candidate first sub-matrix corresponding to each sub-matrix to be updated;

[0040] The candidate first submatrix is ​​transferred from the preset buffer area to the first buffer area of ​​the calculation unit corresponding to each submatrix to be updated, and the target first submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated is obtained from the candidate first submatrix in the first buffer area;

[0041] Determining in the transfer buffer area a candidate second sub-matrix corresponding to each of the sub-matrices to be updated;

[0042] The candidate second submatrix is ​​transferred to a second buffer of a calculation unit corresponding to each submatrix to be updated, and a target second submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated is obtained from the candidate second submatrix in the second buffer.

[0043] In some embodiments, the first buffer includes a first sub-area and a second sub-area, the second buffer includes a third sub-area and a fourth sub-area, and the first determining module is configured to:

[0044] Determine a first idle sub-area in the first sub-area and the second sub-area;

[0045] Moving the candidate first sub-matrix from the preset buffer area to the first idle sub-area;

[0046] The second determination module is used for:

[0047] Determine a second idle sub-area in the third sub-area and the fourth sub-area;

[0048] The candidate second sub-matrix is ​​moved from the preset buffer area to the second idle sub-region.

[0049] In some embodiments, the computing module is used to:

[0050] Inputting the target first submatrix and the target second submatrix into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated to perform matrix multiplication operation, so as to obtain the target submatrix corresponding to each submatrix to be updated;

[0051] Each of the sub-matrices to be updated and the target sub-matrix are input into a vector calculation unit of a calculation unit corresponding to each of the sub-matrices to be updated to perform a vector addition operation, so as to obtain an update sub-matrix corresponding to each of the sub-matrices to be updated.

[0052] In some implementations, after the target first submatrix and the target second submatrix are input into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated to perform a matrix multiplication operation to obtain the target submatrix corresponding to each submatrix to be updated, the calculation module is used to:

[0053] Deleting the target first sub-matrix from the first buffer to update a first free sub-region in the first buffer;

[0054] The target second sub-matrix is ​​deleted from the second buffer to update a second free sub-region in the second buffer.

[0055] In some embodiments, the segmentation module is used to:

[0056] determining a total number of the plurality of computing units;

[0057] The multiple matrices to be updated are evenly distributed to each computing unit according to the total number.

[0058] In order to achieve the above-mentioned purpose, the third aspect of an embodiment of the present application provides a computer-readable storage medium, which stores multiple instructions, and the instructions are suitable for a processor to load to execute the data processing method provided by the embodiment of the present application.

[0059] In order to achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the data processing method provided in the embodiment of the present application when executing the computer program.

[0060] In an embodiment of the present application, a neural network processor includes a global memory, a transfer buffer area and multiple computing units. A plurality of sub-matrices to be updated are obtained by performing block processing on a matrix to be updated, and the plurality of sub-matrices to be updated are evenly distributed to each computing unit; a first sub-matrix corresponding to each sub-matrix to be updated is determined in a first matrix, and the first sub-matrix is ​​moved from the global memory to a preset buffer area of ​​a computing unit corresponding to each sub-matrix to be updated; a second sub-matrix corresponding to each sub-matrix to be updated is determined in a second matrix, and the second sub-matrix is ​​moved from the global memory to the transfer buffer area; a target first sub-matrix requiring a general matrix multiplication operation corresponding to each sub-matrix to be updated is obtained from the preset buffer area, and a target second sub-matrix requiring a general matrix multiplication operation is obtained from the transfer buffer area for each sub-matrix to be updated; a general matrix multiplication operation is performed on each sub-matrix to be updated, a target first sub-matrix corresponding to each sub-matrix to be updated and a target second sub-matrix corresponding to each sub-matrix to be updated according to the computing unit corresponding to each sub-matrix to be updated, to obtain an updated sub-matrix corresponding to each sub-matrix to be updated, and an updated matrix corresponding to the matrix to be updated is generated according to each updated sub-matrix.

[0061] In this way, by performing block processing on the matrix to be updated, a plurality of sub-matrices to be updated are obtained, and the first matrix and the second matrix are processed in blocks to obtain the first sub-matrix and the second sub-matrix corresponding to each sub-matrix to be updated, and then the first sub-matrix and the second sub-matrix to be processed by each computing unit are pre-moved from the global memory to the corresponding cache area, and the number of times each computing unit subsequently performs a general matrix multiplication operation can be reduced, and each computing unit processes the evenly distributed sub-matrix to be updated, the first sub-matrix and the second sub-matrix, which can achieve load balancing when multiple computing units perform general matrix multiplication operations, thereby improving the efficiency of each computing unit when performing general matrix multiplication operations, thereby efficiently exerting the computing power of the neural network processor.

[0062] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0064] Figure 1 is a schematic diagram of the structure of a neural network processor provided in an embodiment of the present application;

[0065] Figure 2 is a schematic diagram of the structure of a computing unit provided in an embodiment of the present application;

[0066] Figure 3 It is a flowchart of a data processing method provided by an embodiment of the present application;

[0067] Figure 4 It is a schematic diagram of matrix block processing provided by an embodiment of the present application;

[0068] Figure 5 is a schematic diagram of matrix grouping provided in an embodiment of the present application;

[0069] Figure 6 is another schematic diagram of matrix grouping provided in an embodiment of the present application;

[0070] Figure 7 is another schematic diagram of matrix grouping provided in an embodiment of the present application;

[0071] Figure 8 is a schematic diagram of the structure of the first buffer zone and the second buffer zone provided in an embodiment of the present application;

[0072] Fig. 9 is another flow chart of the data processing method provided in the embodiment of the present application;

[0073] Fig.10 It is a structural schematic diagram of a data processing device implemented in the present application;

[0074] Fig.11 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0075] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0076] It can be understood that in the specific implementation of this application, the matrix-related data involved, when the above embodiments of this application are applied to specific products or technologies, need to obtain user permission or consent, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards.

[0077] It should be noted that some of the processes described in the specification, claims and the above-mentioned drawings include multiple steps that appear in a specific order, but it should be clearly understood that these steps may not be executed in the order in which they appear in this document or may be executed in parallel. The step numbers are only used to distinguish between different steps, and the numbers themselves do not represent any execution order.

[0078] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations:

[0079] A neural network processor (NPU) is a hardware device specially designed to accelerate neural network calculations. The neural network processor can specifically design matrix calculations, vector calculations, scalar calculations, etc. The neural network processor can implement general matrix multiplication operations between matrices. The neural network processor contains multiple computing units, each of which can implement general matrix multiplication operations between matrices.

[0080] General Matrix Multiply (GEMM) is a basic linear algebra operation used to calculate the product of two matrices. General Matrix Multiply is a very general computational operation that can be applied to a variety of fields and problems. Many scientific computing, engineering applications, and machine learning algorithms involve matrix multiplication, so an efficient implementation of General Matrix Multiply is critical to the performance of these fields. Since the computational complexity of matrix multiplication is usually very large, the performance of General Matrix Multiply has a significant impact on the performance of the entire computing system. Optimizing the implementation of General Matrix Multiply can significantly improve computational efficiency and reduce computing time and resource consumption.

[0081] The above is a detailed explanation of the relevant technical terms.

[0082] First, let's explain the technical problems existing in the relevant technologies:

[0083] With the development of artificial intelligence technology, artificial intelligence-related pending data currently needs to be processed by a specific processor to perform data operations on the pending data. The most commonly used processor is the graphics processing unit (GPU), which can perform matrix operations, vector operations, etc. For example, the graphics processor can perform general matrix multiplication (Gemm, General Matrix Multiplication) operations.

[0084] In the related art, when a graphics processor performs general matrix multiplication operations on a matrix, a relatively efficient algorithm can be used to perform the operation. However, when a neural network processor (NPU) performs general matrix multiplication operations, due to the difference in the computing units of the graphics processor and the neural network processor, the general matrix multiplication-related algorithms applicable to the graphics processor are not applicable to the neural network processor. Although some neural network processors have their own general matrix multiplication algorithms, the general matrix multiplication algorithms they come with cannot efficiently bring into play the computing power of the neural network processor, resulting in a waste of computing power of the neural network processor.

[0085] Therefore, when the neural network processor in the related art performs general matrix multiplication operations, the operation efficiency is low, resulting in the inability to efficiently exert the computing power of the neural network processor.

[0086] Take the Ascend processor as an example. This is a neural network processor. The computing unit of the neural network processor is different from that of the traditional graphics processor. Therefore, the general matrix multiplication-related algorithms applicable to the graphics processor cannot be applied to the neural network processor. Although the neural network processor comes with a general matrix multiplication-related algorithm, the algorithm cannot efficiently exert the computing power of the neural network processor.

[0087] For example, in the neural network processor, the size of input matrix A is m*k, the size of input matrix B is k*n, and the size of input-output matrix C is m*n. It is known that the FP16 theoretical peak performance of the neural network processor is 313TFlops, which means that 313*10^12 floating-point calculations can be performed per second. After the relevant matrices are input into the neural network processor, the test results are shown in Table 1:

[0088] Test matrix size Test performance (TFlops) m=3200k=3200n=3200 162.6 m=6400k=6400n=6400 196.9 m=12800k=12800n=12800 164.5 m=25600k=25600n=25600 171.2

[0089] Table 1

[0090] As shown in Table 1, the best test performance is 196.9TFlops, while the theoretical peak performance of the neural network processor is 313TFlops, and the floating point utilization is about 196.9 / 313*100%=62.9%. In other words, when performing general matrix multiplication operations, about 37% of the computing power is not used, resulting in low efficiency of the neural network processor when performing general matrix multiplication operations.

[0091] To this end, an embodiment of the present application provides a data processing method suitable for a neural network processor, which can improve the computing efficiency of the neural network processor when performing general matrix multiplication operations, thereby efficiently exerting the computing power of the neural network processor. The data processing method, device, computer equipment and storage medium provided by the embodiment of the present application will be described in detail below.

[0092] See also Figure 1 , Figure 1 It is a schematic diagram of the structure of the neural network processor provided in an embodiment of the present application.

[0093] like Figure 1 As shown, the neural network processor includes multiple computing units, and the multiple computing units can perform computing tasks in parallel, such as general matrix multiplication operations on matrices. Each computing unit is connected to the global memory, and each computing unit can obtain matrix and other data from the global memory. Each computing unit can transfer the calculation results to the global memory, such as transferring the general matrix multiplication results to the global memory.

[0094] In some embodiments, each computing unit includes units with different functions, such as a matrix computing unit, a vector computing unit, and a scalar computing unit. These units with different functions can implement related functions. For example, the matrix computing unit is responsible for performing matrix operations, the vector computing unit is responsible for performing vector operations, and the scalar computing unit is responsible for various types of scalar data operations and program flow control.

[0095] Please combine Figure 2 , Figure 2 It is a schematic diagram of the structure of the computing unit provided in the embodiment of the present application.

[0096] The computing unit includes a first buffer, a second buffer, a third buffer, a preset buffer area, a matrix computing unit, a vector computing unit and a unified buffer.

[0097] Among them, the storage areas such as the first buffer area, the second buffer area, the third buffer area and the preset cache area are mainly related to the matrix calculation unit, and the matrix calculation unit stores or reads data in these storage areas.

[0098] For example, the matrix data can be pre-stored in a preset cache area, which acts as a data transfer area and can temporarily store the matrix data used by subsequent matrix calculation units for matrix operations. For example, the matrix data can be obtained from the global memory of the neural network processor and moved to the preset cache area.

[0099] The first buffer can be used to store matrix data that the matrix calculation unit is about to need matrix operation. The matrix data that is about to need matrix operation can be moved from the preset cache area to the first buffer, and then the matrix calculation unit can obtain the matrix data that is about to need matrix operation from the first buffer.

[0100] The second buffer can be used to store another matrix data that the matrix calculation unit is about to need matrix operation. The other matrix data that is about to need matrix operation can be moved from the global memory or the transfer buffer to the second buffer, and then the matrix calculation unit can obtain the other matrix data that is about to need matrix operation from the second buffer.

[0101] The matrix calculation unit can perform a matrix operation, such as a matrix multiplication operation, on the matrix data requiring the matrix operation and another matrix data, so as to obtain a matrix operation result, and then send the matrix operation result to the third buffer.

[0102] The third buffer can store the matrix operation results output by the matrix calculation unit after performing the matrix operation, and the matrix operation results can be sent to the global memory for storage.

[0103] Global memory can be understood as a storage area accessed by all computing units in the entire neural network processor. It usually has a relatively large storage capacity and can store a large amount of data. This makes it suitable for storing entire data sets or large model parameters, etc. Global memory can be accessed by different computing units at any time during the entire program execution process. The access latency of global memory is high because the data needs to be read from a relatively distant storage location, involving the transmission of the memory bus and the search of the storage hierarchy.

[0104] The transfer buffer can be understood as a cache area independent of the global memory and the computing unit. Its function is to reduce the number of times the computing unit accesses the global memory, thereby improving the computing efficiency of the computing unit. For example, the matrix data corresponding to the matrix computing unit can be stored in the transfer buffer, and at least part of the matrix data can be moved from the transfer buffer to the second buffer later.

[0105] A unified buffer generally refers to a specific area or structure used to store and manage data, which can be shared and accessed by multiple components or programs. The purpose of a unified buffer is to improve the efficiency of data storage and access, reduce data copying and movement, and thus optimize system performance. For example, vectors and matrices that need to be calculated by a vector computing unit can be moved from global memory to a unified buffer.

[0106] The vector calculation unit can obtain the matrix and vector that need to perform vector operation from the uniform buffer, and then perform vector calculation, such as vector addition operation, and finally send the vector calculation result to the uniform buffer, and then send the calculation result to the global memory through the uniform buffer.

[0107] The above is a relevant introduction to the entire computing unit. The data processing method provided by the embodiment of the present application will be described in detail below.

[0108] The data processing methods provided in the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, multiprocessor systems, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0109] Please refer to 3, Figure 3 : is a flowchart of a data processing method provided by an embodiment of the present application. The data processing method may include the following steps:

[0110] Step 110, dividing the matrix to be updated into blocks to obtain multiple sub-matrices to be updated, and evenly distributing the multiple sub-matrices to be updated to each computing unit;

[0111] Step 120: determining a first sub-matrix corresponding to each sub-matrix to be updated in the first matrix, and moving the first sub-matrix from the global memory to a preset buffer area of ​​a computing unit corresponding to each sub-matrix to be updated;

[0112] Step 130: determine in the second matrix the second sub-matrix corresponding to each sub-matrix to be updated, and move the second sub-matrix from the global memory to the transfer buffer area;

[0113] Step 140: Obtain a target first submatrix that requires a general matrix multiplication operation corresponding to each submatrix to be updated from a preset buffer area, and obtain a target second submatrix that requires a general matrix multiplication operation corresponding to each submatrix to be updated from a transfer buffer area;

[0114] Step 150: Perform a general matrix multiplication operation on each submatrix to be updated, a target first submatrix corresponding to each submatrix to be updated, and a target second submatrix according to the calculation unit corresponding to each submatrix to be updated, to obtain an update submatrix corresponding to each submatrix to be updated, and generate an update matrix corresponding to the matrix to be updated according to each update submatrix.

[0115] Steps 110 to 150 will be described in detail below.

[0116] In step 110, the matrix to be updated is divided into blocks to obtain a plurality of sub-matrices to be updated, and the plurality of sub-matrices to be updated are evenly distributed to each computing unit.

[0117] In the present application, the neural network processing has multiple computing units, and multiple computing units can perform calculations at the same time, so the matrix to be updated can be divided into multiple sub-matrices to be updated, and then the multiple sub-matrices to be updated are evenly distributed to different computing units, each computing unit corresponds to at least one sub-matrix to be updated, and each computing unit can process the corresponding sub-matrix to be updated. Load balancing of multiple computing units is achieved.

[0118] Subsequently, each computing unit can be used to update the corresponding submatrix to be updated, such as general matrix multiplication operations, to obtain the updated update submatrix, and finally generate the update matrix corresponding to the matrix to be updated according to each update submatrix. Multiple computing units perform calculations simultaneously, achieving load balancing of multiple computing units, thereby giving full play to the computing power of each computing unit, which greatly saves the time for updating the matrix to be updated and more efficiently exerts the computing power of the neural network processing.

[0119] It should be noted that when the update matrix is ​​divided into blocks, the size of the divided sub-matrix to be updated needs to be considered. For example, a preset size can be set, and the size of the sub-matrix to be updated should be smaller than the preset size. This can prevent the size of the sub-matrix to be updated from being too large, resulting in an excessive amount of data in the sub-matrix to be updated. It is also necessary to continue to divide the sub-matrix to be updated into blocks, thereby improving the efficiency of block division.

[0120] In some implementations, the multiple sub-matrices to be updated are evenly distributed to each computing unit, including:

[0121] (1.1) Determine the total number of multiple computing units;

[0122] (1.2) Multiple matrices to be updated are evenly distributed to each computing unit according to the total number.

[0123] Among them, the total number of multiple computing units of the neural network processor can be determined, and then the sub-matrices to be updated can be evenly distributed to each computing unit, for example, the same number of sub-matrices to be updated can be equally distributed to each computing unit, or the difference in the number of sub-matrices to be updated distributed to any two computing units is within a preset number range. In this way, load balancing can be achieved for each computing unit when performing calculations.

[0124] For a more detailed understanding of the process of updating matrix blocks in this application, please refer to Figure 4 , Figure 4 It is a schematic diagram of matrix block processing provided in an embodiment of the present application.

[0125] In the present application, the corresponding calculation formula for each computing unit when performing a general matrix multiplication operation is: C←αAB+βC, where A is the first matrix, B is the second matrix, C on the right side of the formula is the matrix to be updated, C on the left side of the formula is the update matrix, α is the first preset floating point number, and β is the second preset floating point number.

[0126] like Figure 4 As shown, the size of the first matrix is ​​M*K, the size of the second matrix is ​​K*N, and the size of the matrix to be updated is M*N.

[0127] It can be seen from the calculation formula of the general matrix multiplication that the update of the matrix to be updated also needs to be combined with the first matrix and the second matrix. Therefore, it is necessary to process the matrix to be updated in blocks to obtain multiple sub-matrices to be updated. At the same time, it is also necessary to process the first matrix in blocks to obtain the first sub-matrix corresponding to each sub-matrix to be updated. The second matrix is ​​processed in blocks to obtain the second sub-matrix corresponding to each sub-matrix to be updated. Finally, the update of each sub-matrix to be updated is realized through the first sub-matrix and the second sub-matrix corresponding to each sub-matrix to be updated.

[0128] First, the block processing of the matrix to be updated is introduced, and then how to perform block processing on the first matrix and the second matrix is ​​introduced.

[0129] The matrix to be updated is a large matrix, and a partitioning standard can be set within a preset size, such as a size standard of M1*N1. The matrix to be updated can be divided into blocks according to this standard, and the size of most of the sub-matrices to be updated is M1*N1.

[0130] Under this division standard, there may be a situation where the matrix to be updated cannot be evenly divided, and there may be some sub-matrices with sizes smaller than M1*N1. These sub-matrices are also determined as sub-matrices to be updated. Figure 4As shown, the size of the rightmost and bottommost sub-matrices to be updated is smaller than M1*N1.

[0131] It can be seen from the calculation formula of the general matrix multiplication that in order to update the matrix to be updated, the first matrix and the second matrix need to be combined. Therefore, after the matrix to be updated is processed in blocks to obtain multiple sub-matrices to be updated, the first matrix needs to be processed in blocks to obtain the first sub-matrix corresponding to each sub-matrix to be updated. The second matrix is ​​processed in blocks to obtain the second sub-matrix corresponding to each sub-matrix to be updated, and finally the update of each sub-matrix to be updated is realized through the first sub-matrix and the second sub-matrix corresponding to each sub-matrix to be updated.

[0132] In step 120, a first sub-matrix corresponding to each sub-matrix to be updated is determined in the first matrix, and the first sub-matrix is ​​moved from the global memory to a preset buffer area of ​​a computing unit corresponding to each sub-matrix to be updated.

[0133] It is understandable that after the matrix to be updated is divided into blocks, the first matrix also needs to be divided into blocks, so as to obtain the first sub-matrix corresponding to the sub-matrix to be updated in the first matrix. Then the first sub-matrix is ​​moved from the global memory to the preset cache area of ​​the calculation unit corresponding to each sub-matrix to be updated, and the subsequent calculation unit can obtain the first sub-matrix from the preset cache area when performing a general matrix multiplication operation, thereby reducing the number of interactions with the global memory.

[0134] The amount of data stored in the global memory is large, and it generally uses a relatively low-speed storage medium, which results in a long time to search and obtain data in the global memory, and additional data search time is wasted when obtaining data from the global memory. The preset cache area is a high-speed memory inside the computing unit. The first sub-matrix is ​​pre-extracted to the preset cache area. When the subsequent computing unit performs a general matrix multiplication operation, the first sub-matrix can be quickly obtained, and the number of interactions with the global memory can be reduced. Therefore, this can improve the computing efficiency of each computing unit.

[0135] In some implementations, determining in the first matrix a first submatrix corresponding to each submatrix to be updated includes:

[0136] (1.1) determining a first threshold size of a matrix that can be stored in a first buffer of each computing unit, the first buffer being used to obtain and store a target first submatrix that requires a general matrix multiplication operation and corresponds to each submatrix to be updated from a preset cache area;

[0137] (1.2) dividing the first matrix into blocks according to a first threshold size to obtain a plurality of first sub-matrices;

[0138] (1.3) Determine a first sub-matrix corresponding to each sub-matrix to be updated from among the multiple first sub-matrices.

[0139] Among them, each computing unit also includes a first buffer, which is used to obtain and store the target first sub-matrix that requires general matrix multiplication operation corresponding to each sub-matrix to be updated from the preset cache area. It can also be understood that the first sub-matrix that the computing unit is about to perform general matrix multiplication operation is moved from the preset cache area to the first buffer in advance, and the matrix computing unit in the subsequent computing unit can directly obtain the target first sub-matrix that requires general matrix multiplication operation from the first buffer, which can further improve the computing efficiency of the computing unit.

[0140] However, the memory of the first buffer is limited, so it is necessary to determine a first threshold size of the matrix that can be stored in the first buffer, and the first threshold size is the size of the largest matrix that can be cached in the first buffer. Therefore, the size of the first submatrix transferred to the first buffer must be smaller than the first threshold size.

[0141] In the present application, a dual storage area is set for the first buffer. The first sub-matrix in one storage area can be read and calculated by the matrix calculation unit, and the first sub-matrix in the other storage area can wait for the next reading and calculation by the matrix calculation unit. After the first sub-matrix in a storage area of ​​the dual storage area is read and calculated, the data in the storage area can be deleted, and then a new first sub-matrix can be obtained from the preset cache area and stored in the storage area.

[0142] Therefore, in the present application, the first matrix can be divided into blocks according to the first threshold size to obtain multiple first sub-matrices, so that the first sub-matrices meet the dual storage area requirements of the first buffer, for example, the sum of the sizes of the two first sub-matrices added together is smaller than the first threshold size.

[0143] Combination Figure 4 , the size division standard of the first submatrix in this application can be M1*K1. The size of most of the first submatrices obtained is M1*K1. Under this division standard, there may be a situation where the first matrix cannot be evenly divided, and there may be some submatrices with sizes smaller than M1*K1. These submatrices are also determined as the first submatrices. Figure 4 As shown, the size of the first rightmost and bottommost submatrix is ​​smaller than M1*K1.

[0144] In the present application, multiple sub-matrices to be updated, multiple first sub-matrices, and multiple second sub-matrices can be grouped, for example, according to the total number of multiple computing units as a grouping criterion, assuming that the total number is 3, in the multiple sub-matrices to be updated, the multiple sub-matrices to be updated can be divided into multiple groups according to the grouping criterion. When the subsequent computing unit updates the sub-matrix to be updated, the sub-matrix to be updated of each group can be updated in turn, that is, after the sub-matrix to be updated of the previous group is updated, the sub-matrix to be updated of the next group is updated.

[0145] Please combine Figure 5 , Figure 5 It is a schematic diagram of matrix grouping provided in an embodiment of the present application.

[0146] For example, the total number of multiple computing units is used as a grouping standard, assuming that the total number is 3, and the matrix rows are divided to obtain multiple groups corresponding to the sub-matrices to be updated. After the sub-matrices to be updated are grouped, the first sub-matrix and the second sub-matrix need to be grouped according to the grouping of the sub-matrices to be updated.

[0147] Taking a group C1 of submatrices to be updated as an example, the submatrices to be updated of group C1 correspond to group A1 in the first matrix, that is, all first submatrices in group A1 are related to all submatrices to be updated in group C1. The submatrices to be updated of group C1 correspond to groups B1, B2, and B3 in the second matrix, that is, all second submatrices in groups B1, B2, and B3 are related to all submatrices to be updated in group C1. The above-described groups are Figure 5 The middle is the dark colored area.

[0148] Furthermore, since the capacity of the preset buffer area of ​​the computing unit is limited, and the first sub-matrix needs to be pre-stored in the preset buffer area later, a large number of first sub-matrices cannot be stored in the preset buffer area at one time, then the grouping corresponding to the first matrix can be further processed to obtain multiple sub-groups, such as Figure 5 As shown, group A1 is further grouped to obtain subgroups A11, A12 and A13. From each of subgroups A11, A12 and A13, the first submatrix corresponding to each matrix to be updated can be selected and transferred to the preset buffer area of ​​the calculation unit corresponding to each matrix to be updated.

[0149] In the present application, multiple sub-matrices to be updated, multiple first sub-matrices, and multiple second sub-matrices are grouped. They can be divided according to matrix rows or matrix columns, and each sub-matrix to be updated has corresponding row and column information in the entire matrix to be updated.

[0150] exist Figure 5In the grouping shown, the multiple sub-matrices to be updated, the multiple first sub-matrices, and the multiple second sub-matrices are grouped by matrix rows. In the multiple sub-matrices to be updated, each sub-matrix to be updated has corresponding row and column information, in the multiple first sub-matrices, each first sub-matrix has corresponding row and column information, and in the multiple second sub-matrices, each second sub-matrix has corresponding row and column information. The row and column information can specifically be the row and column number, which can be used to locate a specific sub-matrix.

[0151] Please also read Figure 6 and Figure 7 , Figure 6 This is another schematic diagram of the matrix grouping provided in an embodiment of the present application. Figure 7 This is another schematic diagram of matrix grouping provided in an embodiment of the present application.

[0152] From the above content, it can be seen that in the embodiment of the present application, when the update matrix, the first matrix, and the second matrix are processed in blocks, there will be some sub-matrices whose sizes are smaller than the normal block size. These sub-matrices with smaller sizes can be grouped together with the adjacent sub-matrices of normal size.

[0153] like Figure 6 and Figure 7 As shown, in the matrix to be updated, the sub-matrix to be updated with a smaller size and the adjacent sub-matrix to be updated with a normal size are grouped together, and this part of the matrix is ​​the remaining matrix.

[0154] Specifically, the remaining matrices can be grouped in a matrix column manner to obtain group ZC1 and group ZC2. Similarly, by matching the row and column information of the sub-matrix to be updated, the first value matrix, and the second sub-matrix, it is determined that the groups corresponding to group ZC1 are groups XA1, XA2, XA3, YB1, YB2, and YB3. In other words, the first sub-matrix and the second sub-matrix corresponding to the sub-matrix to be updated in group ZC1 can be determined from these groups.

[0155] It is determined that the groups corresponding to the group ZC2 are the groups XA1, XA2, XA3, YB4, YB5, and YB6. That is, the first sub-matrix and the second sub-matrix corresponding to the sub-matrix to be updated in the group ZC2 can be determined in these groups.

[0156] It can be seen from the above content that in the present application, the sub-matrix to be updated, the first sub-matrix, and the second sub-matrix can be associated through row and column information to generate a mapping relationship corresponding to the row and column information, so that when a sub-matrix to be updated is updated, the first sub-matrix and the second sub-matrix that match the sub-matrix to be updated can be determined according to the mapping relationship.

[0157] In some embodiments, a group association relationship between the group of the matrix to be updated, the group of the first matrix, and the group of the second matrix can be set, for example, the C1 group corresponds to the group A1 and the groups B1, B2, B3, and then the mapping relationship between the row and column information of the sub-matrices in the group C1, the group A1, and the groups B1, B2, B3 is established. When a certain sub-matrix to be updated is subsequently updated, the group associated with the sub-matrix to be updated in the first matrix and the second matrix can be searched according to the concerned association relationship, and then the first sub-matrix and the second sub-matrix matching the sub-matrix to be updated can be found in the associated group according to the mapping relationship of the row and column information. In this way, the search efficiency of the sub-matrix can be improved, thereby improving the search speed of the sub-matrix, and finally improving the calculation efficiency of the calculation unit.

[0158] In some implementations, determining a first submatrix corresponding to each submatrix to be updated from a plurality of first submatrices includes:

[0159] (1.3.1) determining target row and column information of each to-be-updated submatrix in the to-be-updated matrix, and determining first row and column information of each first submatrix in the first matrix among the plurality of first submatrices;

[0160] (1.3.2) According to the first row and column information, determine a first sub-matrix that matches the target row and column information of each sub-matrix to be updated from among the multiple first sub-matrices.

[0161] Among them, from the above content, it can be known that each sub-matrix to be updated has corresponding target row and column information in the matrix to be updated, Figure 5 Taking the first submatrix to be updated in the upper left corner of the matrix to be updated as an example, its corresponding row is the first row, and its corresponding column is the first column, then the target row and column information is the first row and the first column.

[0162] Each of the multiple first sub-matrices also has corresponding first row and column information. The rows of each first sub-matrix are determined by the rows of the target row and column information and the first row and column information, and then the two are row-matched, so as to determine that the multiple first sub-matrices of the first row in the first matrix are the first sub-matrices that match the sub-matrix to be updated.

[0163] After determining the first submatrix that matches the target row and column information of each submatrix to be updated, the first submatrix can be moved from the global memory to the preset buffer area of ​​the calculation unit corresponding to each submatrix to be updated. In this way, when the subsequent calculation unit takes the first submatrix for calculation, it can directly take it from the preset buffer area, instead of taking the corresponding first submatrix from the global memory every time it calculates. This can improve the calculation efficiency of the calculation unit.

[0164] In some implementations, moving the first sub-matrix from the global memory to a preset cache area of ​​a computing unit corresponding to each sub-matrix to be updated includes:

[0165] (2.1) Determine the capacity of the preset buffer area of ​​each computing unit corresponding to each sub-matrix to be updated;

[0166] (2.2) According to the capacity and the first row and column information, determine the first sub-matrix that needs to be moved currently in the first sub-matrix that matches the rows and columns of each sub-matrix to be updated;

[0167] (2.3) The first sub-matrix that needs to be moved currently is moved from the global memory to the preset cache area of ​​the computing unit corresponding to each sub-matrix to be updated.

[0168] Among them, the capacity of the preset cache area of ​​each computing unit is limited. For example, the preset cache area can only cache three first sub-matrices at a time. Therefore, the number of first sub-matrices moved from the global memory to the preset cache area can be determined according to the capacity of the preset cache area.

[0169] After determining the number of transfers, the row and column information of each submatrix to be updated and the first row and column information of multiple first submatrices can be matched. As can be seen from the above, the matching can be performed specifically through the mapping relationship corresponding to the row and column information, so as to determine the first submatrix that matches each submatrix to be updated. These first submatrices are the first submatrices that need to be transferred currently, and the number of first submatrices is the same as the number of transfers.

[0170] Then the first sub-matrix that needs to be moved is moved from the global memory to the preset cache area of ​​the calculation unit corresponding to each sub-matrix to be updated. When the subsequent calculation unit obtains the first sub-matrix for calculation, it can obtain the target first sub-matrix that needs the general matrix multiplication operation from the preset cache area, thereby reducing the number of times the target first sub-matrix that needs the general matrix multiplication operation is obtained from the global memory. This can improve the calculation efficiency of the calculation unit.

[0171] Please combine Figure 5 , assuming that the preset cache can only cache three first sub-matrices at a time, the number of transfers is 3, and the sub-matrix to be updated is the sub-matrix to be updated in the first row and the first column, then the corresponding first sub-matrix is ​​the first sub-matrix in the first row of the first matrix, where the first sub-matrix in the first row has three groups, and the three first sub-matrices in the first row of group A11 can be transferred first, and transferred from the global memory to the preset cache. After the first sub-matrix in the preset cache is transferred and read, the first sub-matrix in the first row of group A12 is transferred to the preset cache, and then the next round of general matrix multiplication calculation is started.

[0172] It should be noted that, assuming there are three computing units, the sub-matrix to be updated in the first row of group C1 can be assigned to the first computing unit, the sub-matrix to be updated in the second row of group C1 can be assigned to the second computing unit, and the sub-matrix to be updated in the third row of group C1 can be assigned to the third computing unit. In this way, load balancing of multiple computing units can be achieved, which can further improve computing efficiency.

[0173] After all the sub-matrices to be updated in group C1 are updated, the sub-matrix to be updated in the first row of group C2 can be assigned to the first calculation unit, the sub-matrix to be updated in the second row of group C2 can be assigned to the second calculation unit, and the sub-matrix to be updated in the third row of group C2 can be assigned to the third calculation unit. In this way, the update of the sub-matrix to be updated in each group can be realized in sequence.

[0174] In step 130, a second sub-matrix corresponding to each sub-matrix to be updated is determined in the second matrix, and the second sub-matrix is ​​moved from the global memory to the transfer buffer area.

[0175] It can be seen from the above general matrix multiplication operation that the update of the matrix to be updated also requires a second matrix. For the update of each sub-matrix to be updated, the update can be achieved through the first sub-matrix and the second sub-matrix of each sub-matrix to be updated.

[0176] The second matrix can be processed in blocks, so as to obtain the second submatrix corresponding to the submatrix to be updated in the first matrix. Then the second submatrix is ​​moved from the global memory to the transfer cache area, and the subsequent computing unit can obtain the second submatrix from the transfer cache area when performing general matrix multiplication operations, thereby reducing the number of interactions with the global memory. Compared with the global memory, the transfer cache area has a higher transmission speed and lower latency. Therefore, the second submatrix is ​​pre-stored in the transfer cache area, and the computing unit can quickly obtain the second submatrix when performing matrix multiplication operations, thereby improving computing efficiency.

[0177] In some implementations, determining in the second matrix a second submatrix corresponding to each submatrix to be updated includes:

[0178] (1.1) determining a second threshold size of a matrix that can be stored in a second buffer of each computing unit, the second buffer being used to obtain and store a target second submatrix that requires a general matrix multiplication operation and corresponds to each submatrix to be updated from a transfer buffer;

[0179] (1.2) dividing the second matrix into blocks according to the second threshold size to obtain a plurality of second sub-matrices;

[0180] (1.3) Determine a second sub-matrix corresponding to each sub-matrix to be updated from among the multiple second sub-matrices.

[0181] Among them, each computing unit also includes a second buffer, which is used to obtain and store the target second sub-matrix that requires general matrix multiplication operation corresponding to each sub-matrix to be updated from the preset buffer area. It can also be understood that the second sub-matrix that the computing unit is about to perform general matrix multiplication operation is moved to the second buffer from the transfer buffer area in advance, and the matrix computing unit in the subsequent computing unit can directly obtain the target second sub-matrix that requires general matrix multiplication operation from the second buffer, which can further improve the computing efficiency of the computing unit.

[0182] However, the memory of the second buffer is limited, so it is necessary to determine the second threshold size of the matrix that can be stored in the second buffer, and the second threshold size is the size of the largest matrix that can be cached in the second buffer. Therefore, the size of the second submatrix transferred to the second buffer must be smaller than the second threshold size.

[0183] In the present application, a dual storage area is set for the second buffer. The second sub-matrix in one storage area can be read and calculated by the matrix calculation unit, and the second sub-matrix in the other storage area can wait for the next reading and calculation by the matrix calculation unit. After the second sub-matrix in a storage area of ​​the dual storage area is read and calculated, the data in the storage area can be deleted, and then a new second sub-matrix can be obtained from the preset cache area and stored in the storage area.

[0184] Therefore, in the present application, the second matrix can be divided into blocks according to the second threshold size to obtain multiple second sub-matrices, so that the second sub-matrices meet the dual storage area requirements of the first buffer, for example, the sum of the sizes of the two second sub-matrices added together is smaller than the second threshold size.

[0185] Finally, the target row and column information of each submatrix to be updated in the matrix to be updated is determined, and the second row and column information of each second submatrix in the second matrix is ​​determined, and according to the second row and column information, a second submatrix matching the target row and column information of each submatrix to be updated is determined in the plurality of second submatrices. Figure 5 , assuming that the target row and column information of the submatrix to be updated is the first row and first column of the matrix to be updated, all the second submatrices in the first column of the second matrix are determined as the second submatrix corresponding to the matrix to be updated. Then the second submatrix is ​​moved from the global memory to the transfer buffer area.

[0186] In step 140, a target first submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated is obtained from a preset buffer area, and a target second submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated is obtained from a transfer buffer area.

[0187] In the present application, the first submatrix corresponding to the submatrix to be updated is pre-stored in a preset cache area, and the second submatrix corresponding to the submatrix to be updated is pre-stored in a transit cache area. When the subsequent calculation unit performs calculations, it can obtain the target first submatrix that requires general matrix multiplication operation corresponding to each submatrix to be updated from the preset cache area, and obtain the target second submatrix that requires general matrix multiplication operation corresponding to each submatrix to be updated from the transit cache area.

[0188] In some embodiments, obtaining a target first submatrix that requires a general matrix multiplication operation corresponding to each submatrix to be updated from a preset buffer area, and obtaining a target second submatrix that requires a general matrix multiplication operation corresponding to each submatrix to be updated from a transfer buffer area includes:

[0189] (1.1) determining in a preset buffer area a candidate first sub-matrix corresponding to each sub-matrix to be updated;

[0190] (1.2) moving the candidate first submatrix from the preset buffer area to the first buffer area of ​​the computing unit corresponding to each submatrix to be updated, and obtaining the target first submatrix that requires the general matrix multiplication operation corresponding to each submatrix to be updated from the candidate first submatrix in the first buffer area;

[0191] (1.3) determining in the transfer buffer area a candidate second sub-matrix corresponding to each sub-matrix to be updated;

[0192] (1.4) The candidate second submatrix is ​​transferred to the second buffer of the computing unit corresponding to each submatrix to be updated, and the target second submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated is obtained from the candidate second submatrix in the second buffer.

[0193] There are multiple first sub-matrices in the preset cache area. The multiple first sub-matrices can be sorted first to obtain a sorting result, and then the first sub-matrix to be calculated and used by the matrix calculation unit is determined according to the sorting result, and then these first sub-matrices are determined as candidate first sub-matrices.

[0194] The candidate first submatrix is ​​moved from the preset cache area to the first buffer area of ​​the computing unit corresponding to each submatrix to be updated. In the first buffer area, there may be multiple candidate first submatrices, and the target first submatrix that requires general matrix multiplication operation corresponding to each submatrix to be updated may be selected from the multiple candidate first submatrices according to the sorting result.

[0195] There are multiple second sub-matrices in the transfer buffer area. The multiple second sub-matrices can be sorted first to obtain a sorting result, and then the second sub-matrix to be calculated and used by the matrix calculation unit is determined according to the sorting result, and then these second sub-matrices are determined as candidate second sub-matrices.

[0196] The candidate first submatrix is ​​moved from the preset buffer area to the second buffer area of ​​the computing unit corresponding to each submatrix to be updated. In the second buffer area, there may be multiple candidate second submatrices, and the target second submatrix that requires general matrix multiplication operation and corresponds to each submatrix to be updated may be selected from the multiple candidate second submatrices according to the sorting result.

[0197] Please combine Figure 8 , Figure 8 1 is a schematic diagram of the structure of the first buffer and the second buffer provided in an embodiment of the present application. The first buffer includes a first sub-region and a second sub-region, and the second buffer includes a third sub-region and a fourth sub-region. In the first sub-region and the second sub-region, any sub-region can be used to store the first sub-matrix. In the third sub-region and the fourth sub-region, any sub-region can be used to store the second sub-matrix.

[0198] In some implementations, moving the candidate first sub-matrix from a preset buffer area to a first buffer area of ​​a computing unit corresponding to each sub-matrix to be updated includes:

[0199] (1.2.1) determining a first idle sub-area in the first sub-area and the second sub-area;

[0200] (1.2.2) Moving the candidate first sub-matrix from the preset buffer area to the first idle sub-region.

[0201] If there is no matrix data in any of the first sub-region and the second sub-region, the sub-region may be determined as a first idle sub-region, and then the candidate first sub-matrix is ​​moved from the preset buffer area to the first idle sub-region.

[0202] When the matrix calculation unit reads the target first submatrix that requires general matrix multiplication operation corresponding to the submatrix to be updated from the first subregion or the second subregion, the occupied subregion corresponding to the target first submatrix cannot transmit the candidate first submatrix.

[0203] After the matrix calculation unit reads and calculates the target first submatrix, the data of the occupied sub-region corresponding to the target first submatrix can be deleted. For example, if the occupied sub-region is the first sub-region, the target first submatrix can be deleted from the first sub-region, and the first sub-region is updated to the first free sub-region for storing other candidate first sub-matrices. The matrix calculation unit can read and use the candidate first sub-matrix in the second sub-region.

[0204] In some implementations, moving the candidate second sub-matrix to the second buffer of the computing unit corresponding to each sub-matrix to be updated includes:

[0205] (1.4.1) determining a second idle sub-area in the third sub-area and the fourth sub-area;

[0206] (1.4.2) Moving the candidate second sub-matrix from the preset buffer area to the second free sub-region.

[0207] Among them, in the third sub-region and the fourth sub-region, if any sub-region has no matrix data, the sub-region can be determined as the second idle sub-region, and then the candidate second sub-matrix is ​​moved from the preset buffer area to the second idle sub-region.

[0208] When the matrix calculation unit reads the target second submatrix that requires general matrix multiplication operation corresponding to the submatrix to be updated from the third subregion or the fourth subregion, the occupied subregion corresponding to the target second submatrix cannot transmit the candidate second submatrix.

[0209] After the matrix calculation unit reads and calculates the target second submatrix, the data of the occupied sub-region corresponding to the target second submatrix can be deleted. For example, if the occupied sub-region is the third sub-region, the target second submatrix can be deleted from the third sub-region, and the third sub-region is updated to the second free sub-region for storing other candidate second sub-matrices. The matrix calculation unit can read and use the candidate second sub-matrix in the fourth sub-region.

[0210] As can be seen from the above content, in the present application, dual storage areas are set for the first buffer and the second buffer of the computing unit, so that the first buffer and the second buffer can continuously update and store the sub-matrices to be calculated subsequently, thereby improving the computing efficiency of the computing unit.

[0211] In step 150, a general matrix multiplication operation is performed on each submatrix to be updated, a target first submatrix corresponding to each submatrix to be updated, and a target second submatrix to be updated according to the calculation unit corresponding to each submatrix to be updated, to obtain an update submatrix corresponding to each submatrix to be updated, and an update matrix corresponding to the matrix to be updated is generated according to each update submatrix.

[0212] In the present application, the calculation formula of the general matrix performed by the calculation unit is: C←αAB+βC, wherein A is the first matrix, B is the second matrix, C on the right side of the formula is the matrix to be updated, C on the left side of the formula is the update matrix, α is the first preset floating point number, and β is the second preset floating point number.

[0213] Therefore, when updating a submatrix to be updated, the first submatrix and the second submatrix corresponding to the submatrix to be updated can be determined first, and then the first preset floating-point number, the first submatrix and the second submatrix are multiplied to obtain a first multiplication result, which can be understood as the target submatrix. Then the second preset floating-point number and the submatrix to be updated are multiplied to obtain a second multiplication result. Finally, the first multiplication result and the second multiplication result are added to obtain an updated submatrix corresponding to the submatrix to be updated. An update matrix corresponding to the matrix to be updated can be generated by each update submatrix, for example, each update submatrix is ​​updated to the matrix row and column position of the corresponding matrix to be updated in the matrix to be updated, thereby generating an update matrix corresponding to the matrix to be updated.

[0214] In some implementations, performing a general matrix multiplication operation on each submatrix to be updated, a target first submatrix corresponding to each submatrix to be updated, and a target second submatrix according to a calculation unit corresponding to each submatrix to be updated to obtain an updated submatrix corresponding to each submatrix to be updated includes:

[0215] (1.1) Inputting the target first submatrix and the target second submatrix into the matrix calculation unit corresponding to each submatrix to be updated to perform matrix multiplication operation, so as to obtain the target submatrix corresponding to each submatrix to be updated;

[0216] (1.2) Each sub-matrix to be updated and the target sub-matrix are input into the vector calculation unit of the calculation unit corresponding to each sub-matrix to be updated, and a vector addition operation is performed to obtain an update sub-matrix corresponding to each sub-matrix to be updated.

[0217] From the above content, it can be seen that the calculation unit includes a matrix calculation unit and a vector calculation unit. The target first sub-matrix and the target second sub-matrix can be input into the matrix calculation unit corresponding to each sub-matrix to be updated to perform matrix multiplication operations to obtain the target sub-matrix corresponding to each sub-matrix to be updated.

[0218] Then, each sub-matrix to be updated and the target sub-matrix are input into the vector calculation unit of the calculation unit corresponding to each sub-matrix to be updated to perform vector addition operation, so as to obtain the update sub-matrix corresponding to each sub-matrix to be updated.

[0219] The matrix calculation unit is more suitable for matrix multiplication calculation, while the vector calculation unit is more suitable for vector addition operation. By assigning different calculation steps to different units for calculation, the calculation efficiency of the calculation unit can be improved.

[0220] In some implementations, after inputting the target first submatrix and the target second submatrix into the matrix calculation unit corresponding to each submatrix to be updated to perform a matrix multiplication operation to obtain the target submatrix corresponding to each submatrix to be updated, the method further includes:

[0221] (2.1) deleting the target first sub-matrix from the first buffer to update the first free sub-region in the first buffer;

[0222] (2.2) The target second sub-matrix is ​​deleted from the second buffer to update the second free sub-region in the second buffer.

[0223] It can be understood that after the target first submatrix and the target second submatrix are input into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated for matrix multiplication operation, and the target submatrix corresponding to each submatrix to be updated is obtained, the matrix calculation unit has completed the calculation of the target first submatrix and the target second submatrix, and therefore, the target first submatrix can be deleted from the first buffer to update the first free sub-region in the first buffer. For example, if the target first submatrix occupies the first sub-region, it can be deleted from the first sub-region, thereby updating the first sub-region to the first free sub-region. For example, if the target second submatrix occupies the third sub-region, it can be deleted from the third sub-region, thereby updating the third sub-region to the second free sub-region.

[0224] The advantage of this is that the free sub-areas in the first buffer and the second buffer can be continuously updated, so that the candidate first sub-matrix that needs to be subsequently multiplied by the matrix is ​​input into the first buffer, and the candidate second sub-matrix that needs to be subsequently multiplied by the matrix is ​​input into the second buffer. This improves the data transmission efficiency of the computing unit, thereby improving the computing efficiency of the computing unit.

[0225] It should be noted that in the present application, the matrix to be updated, the first matrix and the second matrix are divided into blocks of reasonable sizes, so that the first buffer of each computing unit can save the first sub-matrix that meets its first threshold size, and the second buffer of each computing unit can save the second sub-matrix that meets its second threshold size. This can improve the data transmission and storage efficiency, so that the matrix calculation unit can process different sub-matrices in a program to improve the calculation efficiency of the entire calculation unit.

[0226] Secondly, by evenly distributing multiple matrices to be updated to each computing unit, load balancing of the computing units can be achieved without wasting the computing power of the computing units, thereby improving the computing efficiency of the computing units.

[0227] Furthermore, before each computing unit performs a general matrix multiplication operation, the first sub-matrix can be pre-stored in a preset cache area and the second sub-matrix can be pre-stored in a transfer cache area to reduce the number of interactions between the computing unit and the global memory, thereby improving the computing efficiency of the computing unit.

[0228] In addition, by setting up dual storage areas in the first buffer and the second buffer of the computing unit, the matrix data to be calculated can be continuously transferred to the first buffer and the second buffer, thereby improving the computing efficiency of the computing unit.

[0229] In the present application, the computing efficiency of each computing unit is improved through the above-mentioned method, that is, the computing efficiency of the computing unit when performing general matrix multiplication operations is improved, thereby improving the computing efficiency of the neural network processor when performing general matrix multiplication operations, and effectively exerting the computing power of the neural network processor.

[0230] As can be seen from the above, in the embodiment of the present application, the neural network processor includes a global memory, a transfer buffer area and multiple computing units, and obtains multiple sub-matrices to be updated by block processing of the matrix to be updated, and the multiple sub-matrices to be updated are evenly distributed to each computing unit; the first sub-matrix corresponding to each sub-matrix to be updated is determined in the first matrix, and the first sub-matrix is ​​moved from the global memory to the preset buffer area of ​​the computing unit corresponding to each sub-matrix to be updated; the second sub-matrix corresponding to each sub-matrix to be updated is determined in the second matrix, and the second sub-matrix is ​​moved from the global memory to the transfer buffer area; the target first sub-matrix corresponding to each sub-matrix to be updated that requires a general matrix multiplication operation is obtained from the preset buffer area, and the target second sub-matrix corresponding to each sub-matrix to be updated that requires a general matrix multiplication operation is obtained from the transfer buffer area; according to the computing unit corresponding to each sub-matrix to be updated, a general matrix multiplication operation is performed on each sub-matrix to be updated, the target first sub-matrix corresponding to each sub-matrix to be updated, and the target second sub-matrix to be updated is obtained, so as to obtain the updated sub-matrix corresponding to each sub-matrix to be updated, and the updated matrix corresponding to the matrix to be updated is generated according to each updated sub-matrix.

[0231] In this way, by performing block processing on the matrix to be updated, a plurality of sub-matrices to be updated are obtained, and the first matrix and the second matrix are processed in blocks to obtain the first sub-matrix and the second sub-matrix corresponding to each sub-matrix to be updated, and then the first sub-matrix and the second sub-matrix to be processed by each computing unit are pre-moved from the global memory to the corresponding cache area, and the number of times each computing unit subsequently performs a general matrix multiplication operation can be reduced, and each computing unit processes the evenly distributed sub-matrix to be updated, the first sub-matrix and the second sub-matrix, which can achieve load balancing when multiple computing units perform general matrix multiplication operations, thereby improving the efficiency of each computing unit when performing general matrix multiplication operations, thereby efficiently exerting the computing power of the neural network processor.

[0232] See also Fig. 9 , Fig. 9 : is another flow chart of the data processing method provided in the embodiment of the present application. The data processing method may include the following steps:

[0233] Step 201, dividing the matrix to be updated into blocks to obtain multiple sub-matrices to be updated, and evenly distributing the multiple sub-matrices to be updated to each computing unit;

[0234] Step 202: determining a first threshold size of a matrix that can be stored in a first buffer of each computing unit, the first buffer being used to obtain and store a target first submatrix that requires a general matrix multiplication operation and corresponds to each submatrix to be updated from a preset cache area;

[0235] Step 203: divide the first matrix into blocks according to the first threshold size to obtain a plurality of first sub-matrices;

[0236] Step 204: determine a first sub-matrix corresponding to each sub-matrix to be updated from among the multiple first sub-matrices;

[0237] Step 205: Determine a second threshold size of a matrix that can be stored in the second buffer of each computing unit, the second buffer being used to obtain and store a target second submatrix that requires a general matrix multiplication operation and corresponds to each submatrix to be updated from the transfer buffer area;

[0238] Step 206: Divide the second matrix into blocks according to the second threshold size to obtain a plurality of second sub-matrices;

[0239] Step 207: Determine, from among the plurality of second sub-matrices, a second sub-matrix corresponding to each sub-matrix to be updated;

[0240] Step 208: Determine in a preset buffer area a candidate first sub-matrix corresponding to each sub-matrix to be updated;

[0241] Step 209: Move the candidate first submatrices from the preset buffer area to the first buffer area of ​​the calculation unit corresponding to each submatrix to be updated, and obtain the target first submatrix that needs general matrix multiplication operation corresponding to each submatrix to be updated from the candidate first submatrix in the first buffer area;

[0242] Step 210: Determine in the transfer buffer area a candidate second sub-matrix corresponding to each sub-matrix to be updated;

[0243] Step 211: move the candidate second submatrix to the second buffer of the calculation unit corresponding to each submatrix to be updated, and obtain the target second submatrix that needs general matrix multiplication operation corresponding to each submatrix to be updated from the candidate second submatrix in the second buffer;

[0244] Step 212: input the target first submatrix and the target second submatrix into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated to perform matrix multiplication operation, so as to obtain the target submatrix corresponding to each submatrix to be updated;

[0245] Step 213: Input each sub-matrix to be updated and the target sub-matrix into the vector calculation unit of the calculation unit corresponding to each sub-matrix to be updated to perform vector addition operation, obtain the update sub-matrix corresponding to each sub-matrix to be updated, and generate the update matrix corresponding to the matrix to be updated according to each update sub-matrix.

[0246] In the above embodiments, the description of each embodiment has its own emphasis. For the part that is not described in detail in a certain embodiment, please refer to the detailed description of the above data processing method, which will not be repeated here.

[0247] See also Fig.10 , Fig.10 The present application implements a structural diagram of providing a data processing device. The data processing device can execute the data processing method described in the above embodiment.

[0248] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0249] like Fig.10 As shown, the data processing device 300 is applied to a neural network processor, the neural network processor includes a global memory, a transfer buffer area and multiple computing units, and the data processing device 300 includes:

[0250] A block division module 310 is used to divide the matrix to be updated into blocks to obtain multiple sub-matrices to be updated, and evenly distribute the multiple sub-matrices to be updated to each computing unit;

[0251] A first determination module 320, configured to determine a first sub-matrix corresponding to each sub-matrix to be updated in the first matrix, and to move the first sub-matrix from the global memory to a preset buffer area of ​​a calculation unit corresponding to each sub-matrix to be updated;

[0252] A second determination module 330 is used to determine the second sub-matrix corresponding to each sub-matrix to be updated in the second matrix, and transfer the second sub-matrix from the global memory to the transfer buffer area;

[0253] An acquisition module 340 is used to acquire a target first submatrix that requires a general matrix multiplication operation corresponding to each submatrix to be updated from a preset buffer area, and acquire a target second submatrix that requires a general matrix multiplication operation corresponding to each submatrix to be updated from a transfer buffer area;

[0254] The calculation module 350 is used to perform a general matrix multiplication operation on each sub-matrix to be updated, a target first sub-matrix corresponding to each sub-matrix to be updated, and a target second sub-matrix according to the calculation unit corresponding to each sub-matrix to be updated, to obtain an update sub-matrix corresponding to each sub-matrix to be updated, and generate an update matrix corresponding to the matrix to be updated according to each update sub-matrix.

[0255] In some implementations, the first determining module 320 is configured to:

[0256] Obtain a first matrix, and perform block processing on the first matrix to obtain a first target matrix corresponding to each submatrix to be updated;

[0257] The first target matrix is ​​grouped to obtain a first sub-matrix corresponding to each sub-matrix to be updated;

[0258] Each first sub-matrix is ​​moved from the global memory to a preset buffer area in the corresponding computing unit.

[0259] In some implementations, the first determination module 320 includes a first determination submodule, a block division submodule, and a second determination submodule;

[0260] A first determination submodule is used to determine a first threshold size of a matrix that can be stored in a first buffer of each computing unit, the first buffer being used to obtain and store a target first submatrix that requires a general matrix multiplication operation corresponding to each submatrix to be updated from a preset cache area;

[0261] A block sub-module, used for performing block processing on the first matrix according to a first threshold size to obtain a plurality of first sub-matrices;

[0262] The second determining submodule is used to determine a first submatrix corresponding to each submatrix to be updated from among the multiple first submatrices.

[0263] In some implementations, the second determining submodule is configured to:

[0264] Determine a first threshold size of a matrix that can be stored in a first buffer of each computing unit, the first buffer being used to obtain and store a target first submatrix that requires a general matrix multiplication operation and corresponds to each submatrix to be updated from a preset cache area;

[0265] Divide the first matrix into blocks according to a first threshold size to obtain a plurality of first sub-matrices;

[0266] A first sub-matrix corresponding to each sub-matrix to be updated is determined from the plurality of first sub-matrices.

[0267] In some implementations, the first determining module 320 is configured to:

[0268] Determine the capacity of a preset buffer area of ​​a computing unit corresponding to each sub-matrix to be updated;

[0269] According to the capacity and the first row and column information, determining the first sub-matrix that needs to be moved currently in the first sub-matrix that matches the rows and columns of each sub-matrix to be updated;

[0270] The first sub-matrix currently required to be transferred is transferred from the global memory to the preset buffer area of ​​the computing unit corresponding to each sub-matrix to be updated.

[0271] In some implementations, the second determination module 330 is configured to:

[0272] Determine a second threshold size of a matrix that can be stored in a second buffer of each computing unit, the second buffer being used to obtain and store a target second submatrix that requires a general matrix multiplication operation and corresponds to each submatrix to be updated from the transfer buffer area;

[0273] Divide the second matrix into blocks according to a second threshold size to obtain a plurality of second sub-matrices;

[0274] A second sub-matrix corresponding to each sub-matrix to be updated is determined from the plurality of second sub-matrices.

[0275] In some implementations, the acquisition module 340 is configured to:

[0276] Determine in a preset buffer area a candidate first sub-matrix corresponding to each sub-matrix to be updated;

[0277] The candidate first submatrix is ​​transferred from the preset buffer area to the first buffer area of ​​the calculation unit corresponding to each submatrix to be updated, and the target first submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated is obtained from the candidate first submatrix in the first buffer area;

[0278] Determine in the transfer buffer area a candidate second sub-matrix corresponding to each sub-matrix to be updated;

[0279] The candidate second submatrix is ​​transferred to the second buffer of the computing unit corresponding to each submatrix to be updated, and the target second submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated is obtained from the candidate second submatrix in the second buffer.

[0280] In some embodiments, the first buffer includes a first sub-region and a second sub-region, the second buffer includes a third sub-region and a fourth sub-region, and the first determining module 320 is configured to:

[0281] Determine a first idle sub-area in the first sub-area and the second sub-area;

[0282] Moving the candidate first sub-matrix from the preset buffer area to the first idle sub-area;

[0283] The second determining module 330 is used to:

[0284] Determine a second idle sub-area in the third sub-area and the fourth sub-area;

[0285] The candidate second sub-matrix is ​​moved from the preset buffer area to the second idle sub-region.

[0286] In some embodiments, the computing module 350 is configured to:

[0287] Inputting the target first submatrix and the target second submatrix into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated to perform matrix multiplication operation, so as to obtain the target submatrix corresponding to each submatrix to be updated;

[0288] Each sub-matrix to be updated and the target sub-matrix are input into the vector calculation unit of the calculation unit corresponding to each sub-matrix to be updated to perform vector addition operation, so as to obtain the update sub-matrix corresponding to each sub-matrix to be updated.

[0289] In some implementations, after the target first submatrix and the target second submatrix are input into the matrix calculation unit corresponding to each submatrix to be updated to perform a matrix multiplication operation to obtain the target submatrix corresponding to each submatrix to be updated, the calculation module 350 is used to:

[0290] Deleting the target first sub-matrix from the first buffer to update a first free sub-region in the first buffer;

[0291] The target second sub-matrix is ​​deleted from the second buffer to update a second free sub-region in the second buffer.

[0292] In some implementations, the block module 310 is configured to:

[0293] determining a total number of the plurality of computing units;

[0294] Multiple matrices to be updated are evenly distributed to each computing unit according to the total number.

[0295] In the above embodiments, the description of each embodiment has its own emphasis. For the part that is not described in detail in a certain embodiment, please refer to the detailed description of the above data processing method, which will not be repeated here.

[0296] In the embodiment of the present application, the neural network processor includes a global memory, a transfer buffer area and a plurality of computing units, a block module 310, which is used to perform block processing on the matrix to be updated to obtain a plurality of sub-matrices to be updated, and evenly distribute the plurality of sub-matrices to be updated to each computing unit; a first determination module 320, which is used to determine the first sub-matrix corresponding to each sub-matrix to be updated in the first matrix, and transfer the first sub-matrix from the global memory to the preset buffer area of ​​the computing unit corresponding to each sub-matrix to be updated; a second determination module 330, which is used to determine the second sub-matrix corresponding to each sub-matrix to be updated in the second matrix, and transfer the second sub-matrix from the global memory to the preset buffer area of ​​the computing unit corresponding to each sub-matrix to be updated; The data are transferred from the storage to the transfer buffer area; an acquisition module 340 is used to obtain the target first submatrix that needs a general matrix multiplication operation corresponding to each submatrix to be updated from the preset buffer area, and obtain the target second submatrix that needs a general matrix multiplication operation corresponding to each submatrix to be updated from the transfer buffer area; a calculation module 350 is used to perform a general matrix multiplication operation on each submatrix to be updated, the target first submatrix corresponding to each submatrix to be updated, and the target second submatrix corresponding to each submatrix to be updated according to the calculation unit corresponding to each submatrix to be updated, to obtain the updated submatrix corresponding to each submatrix to be updated, and generate the updated matrix corresponding to the matrix to be updated according to each updated submatrix.

[0297] In this way, by performing block processing on the matrix to be updated, a plurality of sub-matrices to be updated are obtained, and the first matrix and the second matrix are processed in blocks to obtain the first sub-matrix and the second sub-matrix corresponding to each sub-matrix to be updated, and then the first sub-matrix and the second sub-matrix to be processed by each computing unit are pre-moved from the global memory to the corresponding cache area, and the number of times each computing unit subsequently performs a general matrix multiplication operation can be reduced, and each computing unit processes the evenly distributed sub-matrix to be updated, the first sub-matrix and the second sub-matrix, which can achieve load balancing when multiple computing units perform general matrix multiplication operations, thereby improving the efficiency of each computing unit when performing general matrix multiplication operations, thereby efficiently exerting the computing power of the neural network processor.

[0298] See also Fig.11 , Fig.11 : is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. An embodiment of the present application also provides a computer device, the computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the above data processing method when executing the computer program. The computer device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0299] See also Fig.11 , Fig.11 The hardware structure of a computer device according to another embodiment is shown, and the computer device includes:

[0300] The processor 401 may be a neural network processor, which is used to execute relevant programs to implement the technical solution provided in the embodiment of the present application;

[0301] The memory 402 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 402 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 402, and the processor 401 calls and executes the data processing method of the embodiment of this application;

[0302] Input / output interface 403, used to implement information input and output;

[0303] Communication interface 404, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0304] Bus 405 , which transmits information between various components of the device (e.g., processor 401 , memory 402 , input / output interface 403 , and communication interface 404 );

[0305] The processor 401 , the memory 402 , the input / output interface 403 and the communication interface 404 are connected to each other in communication within the device via the bus 405 .

[0306] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program, and the computer program implements the above-mentioned data processing method when executed by a processor.

[0307] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0308] The data processing method, device, computer equipment and storage medium provided in the embodiments of the present application obtain a plurality of sub-matrices to be updated by performing block processing on a matrix to be updated, and evenly distribute the plurality of sub-matrices to be updated to each computing unit; determine a first sub-matrix corresponding to each sub-matrix to be updated in a first matrix, and move the first sub-matrix from a global memory to a preset cache area of ​​a computing unit corresponding to each sub-matrix to be updated; determine a second sub-matrix corresponding to each sub-matrix to be updated in a second matrix, and move the second sub-matrix from a global memory to a transfer cache area; obtain a target first sub-matrix that requires a general matrix multiplication operation corresponding to each sub-matrix to be updated from the preset cache area, and obtain a target second sub-matrix that requires a general matrix multiplication operation corresponding to each sub-matrix to be updated from the transfer cache area; perform a general matrix multiplication operation on each sub-matrix to be updated, a target first sub-matrix corresponding to each sub-matrix to be updated, and a target second sub-matrix corresponding to each sub-matrix to be updated according to the computing unit corresponding to each sub-matrix to be updated, to obtain an updated sub-matrix corresponding to each sub-matrix to be updated, and generate an updated matrix corresponding to the matrix to be updated according to each updated sub-matrix.

[0309] In this way, by performing block processing on the matrix to be updated, a plurality of sub-matrices to be updated are obtained, and the first matrix and the second matrix are processed in blocks to obtain the first sub-matrix and the second sub-matrix corresponding to each sub-matrix to be updated, and then the first sub-matrix and the second sub-matrix to be processed by each computing unit are pre-moved from the global memory to the corresponding cache area, and the number of times each computing unit subsequently performs a general matrix multiplication operation can be reduced, and each computing unit processes the evenly distributed sub-matrix to be updated, the first sub-matrix and the second sub-matrix, which can achieve load balancing when multiple computing units perform general matrix multiplication operations, thereby improving the efficiency of each computing unit when performing general matrix multiplication operations, thereby efficiently exerting the computing power of the neural network processor.

[0310] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0311] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0312] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0313] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0314] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0315] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0316] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0317] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0318] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0319] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0320] The above is a detailed introduction to a data processing method, device, computer equipment and storage medium provided in the embodiments of the present application. The principles and implementation methods of the present application are described in detail using specific examples herein. The description of the above embodiments is only used to help understand the method and its core idea of ​​the present application. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scopes, but the scope of rights of the embodiments of the present application is not limited thereto. Any modifications, equivalent substitutions and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of rights of the embodiments of the present application.

Claims

1. A data processing method, characterized in that: Applied to a neural network processor, the neural network processor includes a global memory, a transfer buffer area and multiple computing units, the method includes: Performing block processing on the matrix to be updated to obtain multiple sub-matrices to be updated, and evenly distributing the multiple sub-matrices to be updated to each computing unit; Determine in the first matrix a first sub-matrix corresponding to each sub-matrix to be updated, and move the first sub-matrix from the global memory to a preset cache area of ​​a computing unit corresponding to each sub-matrix to be updated; Determine in the second matrix a second sub-matrix corresponding to each of the sub-matrices to be updated, and move the second sub-matrix from the global memory to the transfer buffer area; Obtaining a target first submatrix that requires a general matrix multiplication operation corresponding to each submatrix to be updated from the preset buffer area, and obtaining a target second submatrix that requires a general matrix multiplication operation corresponding to each submatrix to be updated from the transfer buffer area; According to the calculation unit corresponding to each sub-matrix to be updated, a general matrix multiplication operation is performed on each sub-matrix to be updated, the target first sub-matrix corresponding to each sub-matrix to be updated, and the target second sub-matrix to obtain an update sub-matrix corresponding to each sub-matrix to be updated, and an update matrix corresponding to the matrix to be updated is generated according to each update sub-matrix.

2. The data processing method according to claim 1, characterized in that: The step of determining in the first matrix a first sub-matrix corresponding to each sub-matrix to be updated includes: Determine a first threshold size of a matrix that can be stored in a first buffer of each computing unit, the first buffer being used to obtain and store a target first submatrix requiring a general matrix multiplication operation corresponding to each submatrix to be updated from the preset cache area; Performing block processing on the first matrix according to the first threshold size to obtain a plurality of first sub-matrices; A first sub-matrix corresponding to each sub-matrix to be updated is determined from the multiple first sub-matrices.

3. The data processing method according to claim 2, characterized in that: The step of determining, from among the plurality of first sub-matrices, a first sub-matrix corresponding to each sub-matrix to be updated comprises: Determine target row and column information of each to-be-updated sub-matrix in the to-be-updated matrix, and determine first row and column information of each first sub-matrix in the first matrix; According to the first row and column information, a first sub-matrix matching the target row and column information of each sub-matrix to be updated is determined from the multiple first sub-matrices.

4. The data processing method according to claim 3, characterized in that: The step of moving the first sub-matrix from the global memory to a preset buffer area of ​​a computing unit corresponding to each sub-matrix to be updated includes: Determining the capacity of a preset buffer area of ​​a computing unit corresponding to each sub-matrix to be updated; According to the capacity and the first row and column information, determining a first sub-matrix that needs to be moved currently in the first sub-matrixes that match the rows and columns of each sub-matrix to be updated; The first sub-matrix currently required to be transferred is transferred from the global memory to a preset buffer area of ​​the calculation unit corresponding to each sub-matrix to be updated.

5. The data processing method according to claim 1, characterized in that: The determining in the second matrix a second submatrix corresponding to each submatrix to be updated includes: Determining a second threshold size of a matrix that can be stored in a second buffer of each computing unit, wherein the second buffer is used to obtain and store a target second sub-matrix that requires a general matrix multiplication operation and corresponds to each sub-matrix to be updated from the transfer buffer area; Performing block processing on the second matrix according to the second threshold size to obtain a plurality of second sub-matrices; A second sub-matrix corresponding to each of the sub-matrices to be updated is determined from the multiple second sub-matrices.

6. The data processing method according to claim 1, characterized in that: The step of obtaining a target first submatrix that requires a general matrix multiplication operation and corresponds to each submatrix to be updated from the preset buffer area, and obtaining a target second submatrix that requires a general matrix multiplication operation and corresponds to each submatrix to be updated from the transfer buffer area includes: Determining in the preset buffer area a candidate first sub-matrix corresponding to each sub-matrix to be updated; The candidate first submatrix is ​​transferred from the preset buffer area to the first buffer area of ​​the calculation unit corresponding to each submatrix to be updated, and the target first submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated is obtained from the candidate first submatrix in the first buffer area; Determining in the transfer buffer area a candidate second sub-matrix corresponding to each of the sub-matrices to be updated; The candidate second submatrix is ​​transferred to a second buffer of a calculation unit corresponding to each submatrix to be updated, and a target second submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated is obtained from the candidate second submatrix in the second buffer.

7. The data processing method according to claim 6, characterized in that: The first buffer includes a first sub-region and a second sub-region, the second buffer includes a third sub-region and a fourth sub-region, and the moving the candidate first sub-matrix from the preset buffer area to the first buffer of the calculation unit corresponding to each sub-matrix to be updated includes: Determine a first idle sub-area in the first sub-area and the second sub-area; Moving the candidate first sub-matrix from the preset buffer area to the first idle sub-area; The step of transferring the candidate second sub-matrix to the second buffer of the computing unit corresponding to each sub-matrix to be updated includes: Determine a second idle sub-area in the third sub-area and the fourth sub-area; The candidate second sub-matrix is ​​moved from the preset buffer area to the second idle sub-region.

8. The data processing method according to claim 6, characterized in that: The step of performing a general matrix multiplication operation on each submatrix to be updated, the target first submatrix corresponding to each submatrix to be updated, and the target second submatrix according to the calculation unit corresponding to each submatrix to be updated, to obtain an update submatrix corresponding to each submatrix to be updated, includes: Inputting the target first submatrix and the target second submatrix into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated to perform matrix multiplication operation, so as to obtain the target submatrix corresponding to each submatrix to be updated; Each of the sub-matrices to be updated and the target sub-matrix are input into a vector calculation unit of a calculation unit corresponding to each of the sub-matrices to be updated to perform a vector addition operation, so as to obtain an update sub-matrix corresponding to each of the sub-matrices to be updated.

9. The data processing method according to claim 8, characterized in that: After inputting the target first submatrix and the target second submatrix into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated to perform matrix multiplication operation to obtain the target submatrix corresponding to each submatrix to be updated, the method further includes: Deleting the target first sub-matrix from the first buffer to update a first free sub-region in the first buffer; The target second sub-matrix is ​​deleted from the second buffer to update a second free sub-region in the second buffer.

10. The data processing method according to claim 1, characterized in that: The step of evenly distributing the plurality of sub-matrices to be updated to each computing unit comprises: determining a total number of the plurality of computing units; The multiple matrices to be updated are evenly distributed to each computing unit according to the total number.

11. A data processing device, characterized in that: Applied to a neural network processor, the neural network processor includes a global memory, a transfer buffer area and multiple computing units, the device includes: A block division module, used for performing block processing on the matrix to be updated to obtain multiple sub-matrices to be updated, and evenly distributing the multiple sub-matrices to be updated to each computing unit; A first determination module, configured to determine a first sub-matrix corresponding to each sub-matrix to be updated in the first matrix, and move the first sub-matrix from the global memory to a preset cache area of ​​a calculation unit corresponding to each sub-matrix to be updated; A second determination module, configured to determine in the second matrix a second sub-matrix corresponding to each sub-matrix to be updated, and to transfer the second sub-matrix from the global memory to the transfer buffer area; An acquisition module, configured to acquire a target first submatrix that requires a general matrix multiplication operation and corresponds to each submatrix to be updated from the preset buffer area, and acquire a target second submatrix that requires a general matrix multiplication operation and corresponds to each submatrix to be updated from the transfer buffer area; A calculation module is used to perform a general matrix multiplication operation on each sub-matrix to be updated, the target first sub-matrix corresponding to each sub-matrix to be updated, and the target second sub-matrix according to the calculation unit corresponding to each sub-matrix to be updated, to obtain an update sub-matrix corresponding to each sub-matrix to be updated, and to generate an update matrix corresponding to the matrix to be updated according to each update sub-matrix.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the data processing method according to any one of claims 1 to 10.

13. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the data processing method according to any one of claims 1 to 10 is implemented.

Images