Data processing method and apparatus, computer device and storage medium

Abstract

Disclosed in the embodiments of the present application are a data processing method and apparatus, a computer device and a storage medium. A neural network processor comprises a global memory, a transfer buffer and a plurality of computing units. The method comprises: partitioning a matrix to be updated, so as to obtain a plurality of sub-matrices to be updated, and equally allocating the plurality of sub-matrices to be updated to the computing units; determining, in a first matrix, a first sub-matrix corresponding to each sub-matrix to be updated, and transferring the first sub-matrix from the global memory to a preset buffer of the computing unit corresponding to the sub-matrix to be updated; determining, in a second matrix, a second sub-matrix corresponding to each sub-matrix to be updated, and transferring the second sub-matrix from the global memory to the transfer buffer; acquiring target first sub-matrices from the preset buffers, and acquiring target second sub-matrices from the transfer buffer; and performing, by means of each computing unit, general matrix multiplication on each sub-matrix to be updated, the target first sub-matrix and the target second sub-matrix, so as to obtain an updated sub-matrix.

Description

Data processing methods, apparatus, computer equipment and storage media Technical Field

[0001] This application relates to the field of computer technology, specifically to a data processing method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the development of artificial intelligence technology, the data to be processed in the context of artificial intelligence needs to be processed by a specific processor. The most commonly used processor is the graphics processing unit (GPU). The GPU can perform matrix operations, vector operations, and other operations. For example, the GPU can perform general matrix multiplication (Gemm) operations.

[0003] In related technologies, when performing general matrix multiplication operations on graphics processing units (GPUs), relatively efficient algorithms can be used. However, when performing general matrix multiplication operations on neural network processing units (NPUs), the algorithms suitable for GPUs are not suitable for NPUs due to the differences in their computational units. Although some NPUs have built-in general matrix multiplication algorithms, these algorithms cannot efficiently utilize the NPU's computational capabilities, resulting in wasted computing power.

[0004] Therefore, in related technologies, neural network processors have low computational efficiency when performing general matrix multiplication operations, which prevents them from efficiently utilizing the computing power of neural network processors.

[0005] Summary of the Invention

[0006] This application provides a data processing method, apparatus, computer device, and storage medium that can improve the computational efficiency of neural network processors when performing general matrix multiplication operations, thereby efficiently utilizing the computational power of neural network processors.

[0007] To achieve the above objectives, a first aspect of this application provides a data processing method applied to a neural network processor, the neural network processor including global memory, a transit cache, and multiple computing units, the method comprising:

[0008] The matrix to be updated is divided into blocks to obtain multiple sub-matrices to be updated, and these multiple sub-matrices to be updated are evenly distributed to each computing unit.

[0009] In the first matrix, the first submatrix corresponding to each submatrix to be updated is determined, and the first submatrix is ​​moved from the global memory to the preset cache area of ​​the computing unit corresponding to each submatrix to be updated;

[0010] In the second matrix, determine the second submatrix corresponding to each submatrix to be updated, and move the second submatrix from the global memory to the transit cache area;

[0011] Obtain the first target submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated from the preset cache area, and obtain the second target submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated from the transit cache area;

[0012] Perform a general matrix multiplication operation on each submatrix to be updated, the target first submatrix and the target second submatrix corresponding to each submatrix to be updated using the computing unit corresponding to each submatrix to be updated, to obtain the updated submatrix corresponding to each submatrix to be updated, and generate the updated matrix corresponding to the submatrix to be updated based on each updated submatrix.

[0013] To achieve the above objectives, a second aspect of this application provides a data processing apparatus applied to a neural network processor, the neural network processor including a global memory, a transit cache, and multiple computing units, the apparatus comprising:

[0014] The block division module is used to divide the matrix to be updated into multiple sub-matrices to be updated, and to distribute the multiple sub-matrices to be updated evenly to each computing unit.

[0015] The first determining module is used to determine the first submatrix corresponding to each submatrix to be updated in the first matrix, and to move the first submatrix from the global memory to the preset cache area of ​​the computing unit corresponding to each submatrix to be updated;

[0016] The second determining module is used to determine the second submatrix corresponding to each submatrix to be updated in the second matrix, and to move the second submatrix from the global memory to the transit cache area;

[0017] The acquisition module is used to acquire the target first submatrix that requires general matrix multiplication operation corresponding to each submatrix to be updated from the preset cache area, and to acquire the target second submatrix that requires general matrix multiplication operation corresponding to each submatrix to be updated from the transit cache area.

[0018] The calculation module is used to perform a general matrix multiplication operation on each submatrix to be updated, the target first submatrix and the target second submatrix corresponding to each submatrix to be updated according to the calculation unit corresponding to each submatrix to be updated, to obtain the updated submatrix corresponding to each submatrix to be updated, and to generate the updated matrix corresponding to the submatrix to be updated according to each updated submatrix.

[0019] In some implementations, the first determining module is used for:

[0020] Obtain the first matrix, and divide the first matrix into blocks to obtain the first target matrix corresponding to each submatrix to be updated;

[0021] The first target matrix is ​​grouped to obtain the first submatrix corresponding to each submatrix to be updated;

[0022] Each first submatrix is ​​moved from the global memory to the preset cache area in the corresponding computing unit.

[0023] In some implementations, the first determining module includes a first determining submodule, a block submodule, and a second determining submodule;

[0024] The first determining submodule is used to determine the first threshold size of the matrix that the first buffer of each computing unit can store. The first buffer is used to obtain and store the target first submatrix that requires general matrix multiplication operation corresponding to each submatrix to be updated from the preset buffer.

[0025] The block segmentation submodule is used to segment the first matrix according to the first threshold size to obtain multiple first sub-matrices;

[0026] The second determining submodule is used to determine the first submatrix corresponding to each submatrix to be updated among the plurality of first submatrixes.

[0027] In some implementations, the second determining submodule is used for:

[0028] A first threshold size of the matrix that can be stored in the first buffer of each computing unit is determined. The first buffer is used to obtain and store the target first submatrix that requires general matrix multiplication operation corresponding to each submatrix to be updated from the preset buffer.

[0029] The first matrix is ​​divided into blocks according to the first threshold size to obtain multiple first sub-matrices;

[0030] The first submatrix corresponding to each submatrix to be updated is determined from the plurality of first submatrixes.

[0031] In some implementations, the first determining module is used for:

[0032] Determine the capacity of the preset cache area for each computational unit corresponding to the submatrix to be updated;

[0033] Based on the capacity and the first row and column information, the first submatrix that needs to be moved is determined from the first submatrix that matches the row and column of each submatrix to be updated;

[0034] The first submatrix that needs to be moved is moved from the global memory to the preset cache area of ​​the computing unit corresponding to each submatrix to be updated.

[0035] In some implementations, the second determining module is used for:

[0036] A second threshold size of the matrix that can be stored in the second buffer of each computing unit is determined. The second buffer is used to obtain and store the target second submatrix that requires general matrix multiplication operation corresponding to each submatrix to be updated from the transit buffer.

[0037] The second matrix is ​​divided into blocks according to the second threshold size to obtain multiple second sub-matrices;

[0038] The second submatrix corresponding to each submatrix to be updated is determined from the plurality of second submatrixes.

[0039] In some implementations, the acquisition module is used for:

[0040] Determine the candidate first submatrix corresponding to each submatrix to be updated in the preset cache area;

[0041] The candidate first submatrix is ​​moved from the preset cache to the first buffer of the computing unit corresponding to each submatrix to be updated, and the target first submatrix that requires general matrix multiplication operation is obtained from the candidate first submatrix in the first buffer.

[0042] In the transit buffer, determine the candidate second submatrix corresponding to each submatrix to be updated;

[0043] The candidate second submatrix is ​​moved to the second buffer of the computation unit corresponding to each submatrix to be updated, and the target second submatrix requiring general matrix multiplication operation is obtained from the candidate second submatrix in the second buffer.

[0044] In some implementations, the first buffer includes a first sub-region and a second sub-region, the second buffer includes a third sub-region and a fourth sub-region, and the first determining module is configured to:

[0045] A first free sub-region is determined within the first sub-region and the second sub-region;

[0046] The candidate first sub-matrix is ​​moved from the preset cache area to the first free sub-region;

[0047] The second determining module is used for:

[0048] A second free sub-region is determined in the third sub-region and the fourth sub-region;

[0049] The candidate second sub-matrix is ​​moved from the preset cache area to the second free sub-region.

[0050] In some implementations, the computing module is used for:

[0051] The first target submatrix and the second target submatrix are input into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated to perform matrix multiplication, thereby obtaining the target submatrix corresponding to each submatrix to be updated;

[0052] Each submatrix to be updated and the target submatrix are input into the vector calculation unit of the calculation unit corresponding to each submatrix to be updated for vector addition operation to obtain the updated submatrix corresponding to each submatrix to be updated.

[0053] In some implementations, after inputting the target first submatrix and the target second submatrix into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated for matrix multiplication to obtain the target submatrix corresponding to each submatrix to be updated, the calculation module is used to:

[0054] The target first sub-matrix is ​​deleted from the first buffer to update the first free sub-region in the first buffer;

[0055] The target second sub-matrix is ​​deleted from the second buffer to update the second free sub-region in the second buffer.

[0056] In some implementations, the block module is used for:

[0057] Determine the total number of the plurality of computing units;

[0058] The plurality of matrices to be updated are evenly distributed to each computing unit according to the total number.

[0059] To achieve the above objectives, a third aspect of this application provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute the data processing method provided in this application.

[0060] To achieve the above objectives, a fourth aspect of this application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the data processing method provided in this application.

[0061] In this embodiment, the neural network processor includes global memory, a transit cache, and multiple computing units. It obtains multiple sub-matrices to be updated by dividing the matrix to be updated into blocks, and evenly distributes these sub-matrices to each computing unit. A first sub-matrix corresponding to each sub-matrix to be updated is determined in the first matrix, and the first sub-matrix is ​​moved from global memory to a preset cache of the computing unit corresponding to each sub-matrix to be updated. A second sub-matrix corresponding to each sub-matrix to be updated is determined in the second matrix, and the second sub-matrix is ​​moved from global memory to the transit cache. A target first sub-matrix requiring general matrix multiplication is obtained from the preset cache, and a target second sub-matrix requiring general matrix multiplication is obtained from the transit cache. General matrix multiplication is performed on each sub-matrix to be updated, its target first sub-matrix, and its target second sub-matrix according to the computing unit corresponding to each sub-matrix to be updated, resulting in an updated sub-matrix corresponding to each sub-matrix to be updated. An updated matrix corresponding to the sub-matrix to be updated is then generated based on each updated sub-matrix.

[0062] Therefore, by dividing the matrix to be updated into blocks, multiple sub-matrices to be updated are obtained. The first matrix and the second matrix are also divided into blocks to obtain the first sub-matrix and the second sub-matrix corresponding to each sub-matrix to be updated. The first sub-matrix and the second sub-matrix to be processed by each computing unit are pre-moved from global memory to the corresponding cache area. This reduces the number of times data is moved from global memory when each computing unit performs general matrix multiplication. Furthermore, the balanced distribution of the sub-matrix to be updated, the first sub-matrix, and the second sub-matrix processed by each computing unit enables load balancing when multiple computing units perform general matrix multiplication, thereby improving the efficiency of each computing unit in performing general matrix multiplication and thus efficiently utilizing the computing power of the neural network processor.

[0063] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0065] Figure 1 is a schematic diagram of the structure of the neural network processor provided in an embodiment of this application;

[0066] Figure 2 is a schematic diagram of the structure of the computing unit provided in an embodiment of this application;

[0067] Figure 3 is a flowchart illustrating the data processing method provided in an embodiment of this application;

[0068] Figure 4 is a schematic diagram of matrix block processing provided in an embodiment of this application;

[0069] Figure 5 is a schematic diagram of matrix grouping provided in an embodiment of this application;

[0070] Figure 6 is another schematic diagram of matrix grouping provided in an embodiment of this application;

[0071] Figure 7 is another schematic diagram of matrix grouping provided in an embodiment of this application;

[0072] Figure 8 is a schematic diagram of the structure of the first buffer and the second buffer provided in an embodiment of this application;

[0073] Figure 9 is another flowchart illustrating the data processing method provided in an embodiment of this application;

[0074] Figure 10 is a schematic diagram of the structure of the data processing apparatus provided in this application;

[0075] Figure 11 is a schematic diagram of the structure of the computer device provided in an embodiment of this application. Detailed Implementation

[0076] To enable those skilled in the art to better understand the solutions of this application, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0077] It is understood that, in the specific embodiments of this application, the matrix-related data involved, when applied to specific products or technologies, requires user permission or consent, and the collection, use and processing of the related data must comply with relevant laws, regulations and standards.

[0078] It should be noted that some processes described in the specification, claims and above figures include multiple steps that appear in a specific order. However, it should be clearly understood that these steps may not be performed in the order they appear in this document or may be performed in parallel. The step numbers are only used to distinguish the different steps and do not represent any execution order.

[0079] Before providing a further detailed description of the embodiments of this application, the nouns and terms used in the embodiments of this application are explained, and the nouns and terms used in the embodiments of this application shall be interpreted as follows:

[0080] A Neural Network Processor (NPU) is a hardware device specifically designed to accelerate neural network computations. NPUs can be designed for matrix computation, vector computation, scalar computation, and more. They can perform general matrix multiplication operations. A NPU contains multiple computational units, each capable of performing general matrix multiplication operations.

[0081] General matrix multiplication (GEMM) is a fundamental linear algebra operation used to compute the product of two matrices. It is a highly versatile computational operation applicable to a wide range of fields and problems. Many scientific computing, engineering applications, and machine learning algorithms involve matrix multiplication; therefore, an efficient implementation of GEMM is crucial for performance in these domains. Since matrix multiplication is typically computationally very expensive, the performance of GEMM has a significant impact on the overall performance of the computing system. Optimizing the implementation of GEMM can significantly improve computational efficiency and reduce computation time and resource consumption.

[0082] The above is a detailed explanation of the relevant technical terms.

[0083] First, let me explain the technical problems existing in the relevant technologies:

[0084] With the development of artificial intelligence technology, the data to be processed in the context of artificial intelligence needs to be processed by a specific processor. The most commonly used processor is the graphics processing unit (GPU). The GPU can perform matrix operations, vector operations, and other operations. For example, the GPU can perform general matrix multiplication (Gemm) operations.

[0085] In related technologies, when performing general matrix multiplication operations on graphics processing units (GPUs), relatively efficient algorithms can be used. However, when performing general matrix multiplication operations on neural network processing units (NPUs), the algorithms suitable for GPUs are not suitable for NPUs due to the differences in their computational units. Although some NPUs have built-in general matrix multiplication algorithms, these algorithms cannot efficiently utilize the NPU's computational capabilities, resulting in wasted computing power.

[0086] Therefore, in related technologies, neural network processors have low computational efficiency when performing general matrix multiplication operations, which prevents them from efficiently utilizing the computing power of neural network processors.

[0087] Taking the Ascend processor as an example, this is a neural network processor. The computing units of this neural network processor differ from those of traditional graphics processing units (GPUs). Therefore, general matrix multiplication algorithms suitable for GPUs are not applicable to this neural network processor. Although the neural network processor has its own general matrix multiplication algorithms, these algorithms cannot efficiently utilize the processor's computing power.

[0088] For example, in this neural network processor, the size of input matrix A is m*k, the size of input matrix B is k*n, and the size of input-output matrix C is m*n. The theoretical peak performance of this neural network processor in FP16 is known to be 313 TFlops, representing 313 * 10^12 floating-point calculations per second. After inputting the relevant matrices into the neural network processor, the test results are shown in Table 1.

[0089] Table 1

[0090] As shown in Table 1, the best test performance is 196.9 TFlops, while the theoretical peak performance of the neural network processor is 313 TFlops. The floating-point utilization is approximately 196.9 / 313*100% = 62.9%. In other words, about 37% of the computing power is not utilized when performing general matrix multiplication operations, resulting in low efficiency of the neural network processor in general matrix multiplication operations.

[0091] Therefore, this application provides a data processing method suitable for neural network processors, which can improve the computational efficiency of neural network processors when performing general matrix multiplication operations, thereby efficiently utilizing the computing power of neural network processors. The data processing method, apparatus, computer equipment, and storage medium provided in this application will be described in detail below.

[0092] Please refer to Figure 1, which is a schematic diagram of the structure of the neural network processor provided in an embodiment of this application.

[0093] As shown in Figure 1, the neural network processor contains multiple computing units that can execute computational tasks in parallel, such as performing general matrix multiplication. Each computing unit is connected to global memory, and can retrieve matrix data from global memory. Each computing unit can also transfer computation results to global memory, such as transferring the result of a general matrix multiplication.

[0094] In some implementations, each computing unit contains units with different functions, such as matrix computing units, vector computing units, and scalar computing units. These units with different functions can perform related functions. For example, the matrix computing unit is responsible for performing matrix operations, the vector computing unit is responsible for performing vector operations, and the scalar computing unit is responsible for various types of scalar data operations and program flow control.

[0095] Please refer to Figure 2, which is a schematic diagram of the structure of the computing unit provided in the embodiment of this application.

[0096] The computation unit includes a first buffer, a second buffer, a third buffer, a preset buffer, a matrix computation unit, a vector computation unit, and a unified buffer.

[0097] Among them, the first buffer, the second buffer, the third buffer, and the preset buffer are mainly related to the matrix calculation unit, which stores or retrieves data in these storage areas.

[0098] For example, matrix data can be pre-stored in a preset cache area. The preset cache area acts as a data transfer area, which can temporarily store matrix data for subsequent matrix calculation units to use for matrix operations. For example, matrix data can be obtained from the global memory of the neural network processor and moved to the preset cache area.

[0099] The first buffer can be used to store matrix data that the matrix calculation unit will need to perform matrix operations on. The matrix data that will need to perform matrix operations can be moved from the preset buffer to the first buffer, and then the matrix calculation unit can obtain the matrix data that will need to perform matrix operations from the first buffer.

[0100] The second buffer can be used to store another matrix data that the matrix calculation unit will need to perform matrix operations on. The other matrix data that will need to perform matrix operations can be moved from global memory or a transfer buffer to the second buffer, and then the matrix calculation unit can obtain the other matrix data that will need to perform matrix operations from the second buffer.

[0101] The matrix calculation unit can perform matrix operations on the matrix data that needs to be performed and another matrix data, such as matrix multiplication, to obtain the matrix operation result, and then send the matrix operation result to the third buffer.

[0102] The third buffer can store the matrix operation results output by the matrix calculation unit after performing matrix operations, and can send the matrix operation results to global memory for storage.

[0103] Global memory can be understood as a storage area accessed by all computational units in the entire neural network processor. It typically has a relatively large storage capacity, capable of storing large amounts of data. This makes it suitable for storing entire datasets or large model parameters. Global memory can be accessed by different computational units at any time throughout the program's execution. However, global memory access latency is relatively high because data needs to be read from relatively distant storage locations, involving memory bus transfers and lookups within the storage hierarchy.

[0104] A transit buffer can be understood as a cache area independent of global memory and computation units. Its function is to reduce the number of times computation units access global memory, thereby improving the computational efficiency of the computation units. For example, the matrix data corresponding to the matrix computation unit can be stored in the transit buffer, and at least part of the matrix data can be moved from the transit buffer to the second buffer later.

[0105] A unified buffer typically refers to a specific area or structure used for storing and managing data, which can be shared and accessed by multiple components or programs. The purpose of a unified buffer is to improve data storage and access efficiency, reduce data copying and movement, and thus optimize system performance. For example, vectors and matrices that a vector computation unit needs to calculate can be moved from global memory to a unified buffer.

[0106] The vector computation unit can obtain the matrices and vectors that need to be used for vector operations from the unified buffer, then perform vector computations, such as vector addition, and finally send the vector computation results to the unified buffer, and then send the computation results to global memory through the unified buffer.

[0107] The above is a brief introduction to the entire computing unit. The data processing method provided in the embodiments of this application will be described in detail below.

[0108] The data processing methods provided in this application can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include personal computers, server computers, multiprocessor systems, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0109] Please refer to Figure 3, which is a flowchart illustrating a data processing method provided in an embodiment of this application. This data processing method may include the following steps:

[0110] Step 110: Divide the matrix to be updated into multiple sub-matrices to be updated, and distribute the multiple sub-matrices to be updated evenly to each computing unit;

[0111] Step 120: Determine the first submatrix corresponding to each submatrix to be updated in the first matrix, and move the first submatrix from global memory to the preset cache area of ​​the computing unit corresponding to each submatrix to be updated;

[0112] Step 130: Determine the second submatrix corresponding to each submatrix to be updated in the second matrix, and move the second submatrix from global memory to the intermediate cache area;

[0113] Step 140: Obtain the first target submatrix that requires general matrix multiplication operation for each submatrix to be updated from the preset buffer, and obtain the second target submatrix that requires general matrix multiplication operation for each submatrix to be updated from the transit buffer.

[0114] Step 150: Perform a general matrix multiplication operation on each submatrix to be updated, the target first submatrix and the target second submatrix corresponding to each submatrix to be updated according to the computing unit corresponding to each submatrix to be updated, to obtain the updated submatrix corresponding to each submatrix to be updated, and generate the updated matrix corresponding to the submatrix to be updated according to each updated submatrix.

[0115] Steps 110 to 150 will be described in detail below.

[0116] In step 110, the matrix to be updated is divided into blocks to obtain multiple sub-matrices to be updated, and these multiple sub-matrices are evenly distributed to each computing unit.

[0117] In this application, the neural network processing has multiple computing units that can perform calculations simultaneously. Therefore, the matrix to be updated can be divided into multiple sub-matrices to be updated, and these sub-matrices are then evenly distributed among different computing units. Each computing unit corresponds to at least one sub-matrix to be updated, and each computing unit can process its corresponding sub-matrix. This achieves load balancing among the multiple computing units.

[0118] Subsequently, each computational unit can update the corresponding submatrix to be updated, such as through general matrix multiplication, to obtain the updated submatrix. Finally, the update matrix corresponding to the matrix to be updated is generated based on each updated submatrix. Multiple computational units perform calculations simultaneously, achieving load balancing across multiple computational units and fully utilizing the computing power of each unit. This greatly saves the time required to update the matrix to be updated, and more efficiently leverages the computational power of the neural network.

[0119] It should be noted that when dividing the matrix to be updated into blocks, the size of the sub-matrices to be updated needs to be considered. For example, a preset size can be set, and the size of the sub-matrices to be updated should be smaller than the preset size. This can prevent the size of the sub-matrices to be updated from being too large, which would result in an excessive amount of data in the sub-matrices to be updated, requiring further division into blocks, thus improving the efficiency of block division.

[0120] In some implementations, multiple submatrices to be updated are evenly distributed to each computational unit, including:

[0121] (1.1) Determine the total number of multiple computing units;

[0122] (1.2) Distribute the multiple matrices to be updated evenly to each computing unit according to the total number.

[0123] This involves determining the total number of computing units in the neural network processor, and then evenly distributing the sub-matrices to be updated to each computing unit. For example, each computing unit can be assigned the same number of sub-matrices, or the difference in the number of sub-matrices assigned to any two computing units can be within a preset range. This achieves load balancing for each computing unit during computation.

[0124] To understand the process of updating matrix blocks in this application in more detail, please refer to Figure 4, which is a schematic diagram of matrix block processing provided in an embodiment of this application.

[0125] In this application, the calculation formula for each computing unit when performing a general matrix multiplication operation is: C←αAB+βC. Where A is the first matrix, B is the second matrix, C on the right side of the formula is the matrix to be updated, C on the left side of the formula is the updated matrix, α is a first preset floating-point number, and β is a second preset floating-point number.

[0126] As shown in Figure 4, the size of the first matrix is ​​M*K, the size of the second matrix is ​​K*N, and the size of the matrix to be updated is M*N.

[0127] As the general formula for matrix multiplication shows, updating the matrix to be updated requires combining the first and second matrices. Therefore, the matrix to be updated needs to be divided into blocks to obtain multiple sub-matrices to be updated. Simultaneously, the first matrix also needs to be divided into blocks to obtain the first sub-matrix corresponding to each sub-matrix to be updated. The second matrix is ​​then divided into blocks to obtain the second sub-matrix corresponding to each sub-matrix to be updated. Finally, each sub-matrix to be updated is updated using its corresponding first and second sub-matrices.

[0128] First, we will introduce the block processing of the matrix to be updated, and then we will introduce how to perform block processing on the first matrix and the second matrix.

[0129] The matrix to be updated is a large matrix. A partitioning standard can be set within a preset size, such as a size standard of M1*N1. The matrix to be updated can be divided into blocks using this standard, and most of the sub-matrices to be updated will have a size of M1*N1.

[0130] Under this partitioning criterion, there may be cases where the matrix to be updated cannot be uniformly divided, resulting in some submatrices having a size smaller than M1*N1. These submatrices are also identified as submatrices to be updated. As shown in Figure 4, the rightmost and bottommost submatrices to be updated have a size smaller than M1*N1.

[0131] As the general formula for matrix multiplication shows, updating the matrix to be updated requires combining the first matrix and the second matrix. Therefore, after dividing the matrix to be updated into blocks to obtain multiple sub-matrices to be updated, the first matrix also needs to be divided into blocks to obtain the first sub-matrix corresponding to each sub-matrix to be updated. The second matrix is ​​then divided into blocks to obtain the second sub-matrix corresponding to each sub-matrix to be updated. Finally, each sub-matrix to be updated is updated using both the first and second sub-matrices corresponding to it.

[0132] In step 120, the first submatrix corresponding to each submatrix to be updated is determined in the first matrix, and the first submatrix is ​​moved from global memory to the preset cache area of ​​the computing unit corresponding to each submatrix to be updated.

[0133] Understandably, after dividing the matrix to be updated into blocks, the first matrix also needs to be divided into blocks to obtain the first submatrix corresponding to the submatrix to be updated. Then, the first submatrix is ​​moved from global memory to the preset cache area of ​​the computation unit corresponding to each submatrix to be updated. When subsequent computation units perform general matrix multiplication operations, they can retrieve the first submatrix from the preset cache area, thereby reducing the number of interactions with global memory.

[0134] Global memory stores large amounts of data and typically uses relatively slow storage media, resulting in significant time spent searching for and retrieving data from it. This additional time is wasted. In contrast, the pre-set cache is a high-speed memory within each computation unit. By pre-extracting the first submatrix into the pre-set cache, subsequent computation units can quickly access the first submatrix during general matrix multiplication operations. This also reduces the number of interactions with global memory, thus improving the computational efficiency of each unit.

[0135] In some implementations, determining the first submatrix corresponding to each submatrix to be updated in the first matrix includes:

[0136] (1.1) Determine the first threshold size of the matrix that the first buffer of each computing unit can store. The first buffer is used to retrieve and store the target first submatrix that requires general matrix multiplication operation corresponding to each submatrix to be updated from the preset buffer.

[0137] (1.2) Divide the first matrix into blocks according to the first threshold size to obtain multiple first sub-matrices;

[0138] (1.3) Determine the first submatrix corresponding to each submatrix to be updated among multiple first submatrixes.

[0139] Each computing unit also includes a first buffer. The first buffer is used to retrieve and store the target first submatrix that needs to be performed on the general matrix multiplication operation for each submatrix to be updated from the preset buffer. In other words, the first submatrix that the computing unit will perform the general matrix multiplication operation on is moved from the preset buffer to the first buffer in advance. The matrix computing units in the subsequent computing units can directly retrieve the target first submatrix that needs to be performed on the general matrix multiplication operation from the first buffer, which can further improve the computing efficiency of the computing unit.

[0140] However, the memory of the first buffer is limited, so it is necessary to determine a first threshold size of the matrix that the first buffer can store. The first threshold size is the largest matrix size that the first buffer can cache. Therefore, the size of the first submatrix moved to the first buffer must be smaller than this first threshold size.

[0141] In this application, a dual storage area is set for the first buffer. The first submatrix in one storage area can be read and calculated by the matrix calculation unit, while the first submatrix in the other storage area can wait for the next read and calculation by the matrix calculation unit. After the first submatrix in one of the dual storage areas is read and calculated, the data in that storage area can be deleted, and then a new first submatrix can be obtained from the preset buffer and stored in that storage area.

[0142] Therefore, in this application, the first matrix can be divided into blocks according to the first threshold size to obtain multiple first sub-matrices, so that the first sub-matrices meet the dual storage area requirements of the first buffer, for example, the sum of the sizes of the two first sub-matrices is less than the first threshold size.

[0143] Referring to Figure 4, the size division criterion for the first submatrix in this application can be M1*K1. Most of the resulting first submatrixes have a size of M1*K1. Under this criterion, there may be cases where the first matrix cannot be uniformly divided, resulting in some submatrixes having a size smaller than M1*K1. These submatrixes are also determined as first submatrixes. As shown in Figure 4, the rightmost and bottommost first submatrixes have a size smaller than M1*K1.

[0144] In this application, multiple sub-matrices to be updated, multiple first sub-matrices, and multiple second sub-matrices can be grouped. For example, the total number of multiple computing units can be used as the grouping criterion. Assuming the total number is 3, the multiple sub-matrices to be updated can be divided into multiple groups according to the grouping criterion. When subsequent computing units update the sub-matrices to be updated, the sub-matrices to be updated in each group can be updated sequentially. That is, after updating the sub-matrices to be updated in the previous group, the sub-matrices to be updated in the next group are updated.

[0145] Please refer to Figure 5, which is a schematic diagram of matrix grouping provided in an embodiment of this application.

[0146] For example, the total number of computational units can be used as the grouping criterion. Assuming the total number is 3, the matrix can be divided by rows to obtain multiple groups corresponding to the submatrix to be updated. After grouping the submatrix to be updated, it is also necessary to group the first submatrix and the second submatrix according to the grouping of the submatrix to be updated.

[0147] Taking a group C1 of the submatrix to be updated as an example, the submatrix to be updated in group C1 corresponds to group A1 in the first matrix. That is, all first submatrices in group A1 are related to all submatrices to be updated in group C1. The submatrices to be updated in group C1 correspond to groups B1, B2, and B3 in the second matrix. That is, all second submatrices in groups B1, B2, and B3 are related to all submatrices to be updated in group C1. The groups described above are represented by dark-colored areas in Figure 5.

[0148] Furthermore, since the capacity of the preset cache of the computing unit is limited, and the first submatrix needs to be pre-stored in the preset cache, a large number of the first submatrixes cannot be stored in the preset cache at once. Therefore, the group corresponding to the first matrix can be further grouped to obtain multiple subgroups, as shown in Figure 5. Group A1 is further grouped to obtain subgroups A11, A12, and A13. From each subgroup A11, A12, and A13, the first submatrix corresponding to each matrix to be updated can be selected and moved to the preset cache of the computing unit corresponding to each matrix to be updated.

[0149] In this application, multiple sub-matrices to be updated, multiple first sub-matrices, and multiple second sub-matrices are grouped. The grouping can be done by matrix rows or matrix columns. Each sub-matrice to be updated has corresponding row and column information in the entire matrix to be updated.

[0150] In the grouping shown in Figure 5, the multiple sub-matrices to be updated, the multiple first sub-matrices, and the multiple second sub-matrices are grouped by matrix rows. Each sub-matrice to be updated has corresponding row and column information; each first sub-matrice has corresponding row and column information; and each second sub-matrice has corresponding row and column information. The row and column information can specifically be the row and column number, which can be used to locate a specific sub-matrix.

[0151] Please refer to Figures 6 and 7 together. Figure 6 is another schematic diagram of matrix grouping provided in an embodiment of this application. Figure 7 is yet another schematic diagram of matrix grouping provided in an embodiment of this application.

[0152] As can be seen from the above, in the embodiments of this application, when performing block processing on the matrix to be updated, the first matrix, and the second matrix, there will be some sub-matrices whose size is smaller than the normal block size. These smaller sub-matrices can be grouped together with the adjacent normal-sized sub-matrices.

[0153] As shown in Figures 6 and 7, in the matrix to be updated, the smaller submatrix to be updated and the neighboring normal-sized submatrix to be updated are grouped together, and this part of the matrix is ​​the remaining part of the matrix.

[0154] Specifically, the remaining matrix can be grouped using matrix columns to obtain group ZC1 and group ZC2. Similarly, by matching the row and column information of the submatrix to be updated, the first submatrix, and the second submatrix, the groups corresponding to group ZC1 are determined to be groups XA1, XA2, XA3, YB1, YB2, and YB3. In other words, the first and second submatrix corresponding to the submatrix to be updated in group ZC1 can be determined from these groups.

[0155] The groups corresponding to group ZC2 are identified as XA1, XA2, XA3, YB4, YB5, and YB6. In other words, the first and second submatrices corresponding to the submatrices to be updated in group ZC2 can be determined from these groups.

[0156] As can be seen from the above, in this application, the submatrix to be updated, the first submatrix, and the second submatrix can be associated with each other through row and column information to generate a mapping relationship corresponding to the row and column information. Thus, when updating a certain submatrix to be updated, the first submatrix and the second submatrix that match the submatrix to be updated can be determined according to the mapping relationship.

[0157] In some implementations, grouping relationships can be established between the groups of the matrix to be updated, the groups of the first matrix, and the groups of the second matrix. For example, group C1 corresponds to groups A1 and B1, B2, and B3. Then, a mapping relationship of row and column information can be established between the submatrices in groups C1, A1, B1, B2, and B3. When updating a submatrix to be updated subsequently, the groups associated with the submatrix to be updated in the first and second matrices can be found first based on the relevant relationships. Then, the first and second submatrices matching the submatrix to be updated can be found in the associated groups based on the mapping relationship of row and column information. This improves the efficiency of submatrix lookup, thereby increasing the lookup speed and ultimately improving the computational efficiency of the computing unit.

[0158] In some implementations, the first submatrix corresponding to each submatrix to be updated is determined among a plurality of first submatrixes, including:

[0159] (1.3.1) Determine the target row and column information of each submatrix to be updated in the matrix to be updated, and determine the first row and column information of each first submatrix in the first matrix;

[0160] (1.3.2) Based on the first row and column information, determine the first submatrix that matches the target row and column information of each submatrix to be updated among multiple first submatrixes.

[0161] As can be seen from the above, each submatrix to be updated has corresponding target row and column information in the matrix to be updated. Taking the first submatrix to be updated in the upper left corner of the matrix to be updated in Figure 5 as an example, its corresponding row is the first row and its corresponding column is the first column. Therefore, the target row and column information is the first row and the first column.

[0162] Each of the multiple first submatrices also has corresponding first row and column information. By using the row information of the target row and column information, and the first row and column information, the row of each first submatrix is ​​determined. Then, row matching is performed between the two to determine the multiple first submatrices in the first row of the first matrix that match the first submatrix to be updated.

[0163] After determining the first submatrix that matches the target row and column information of each submatrix to be updated, the first submatrix can be moved from global memory to the preset cache of the computation unit corresponding to each submatrix to be updated. This way, when subsequent computation units retrieve the first submatrix for computation, they can directly retrieve it from the preset cache, instead of retrieving the corresponding first submatrix from global memory every time a computation is performed. This improves the computational efficiency of the computation units.

[0164] In some implementations, moving the first submatrix from global memory to a preset cache area of ​​the computation unit corresponding to each submatrix to be updated includes:

[0165] (2.1) Determine the capacity of the preset buffer area for each computational unit corresponding to the submatrix to be updated;

[0166] (2.2) Based on the capacity and the first row and column information, determine the first submatrix that needs to be moved in the first submatrix that matches the row and column of each submatrix to be updated;

[0167] (2.3) Move the first submatrix that needs to be moved from the global memory to the preset cache area of ​​the computing unit corresponding to each submatrix to be updated.

[0168] The capacity of the preset cache area for each computing unit is limited. For example, the preset cache area can only cache three first sub-matrices at a time. Therefore, the number of first sub-matrices moved from global memory to the preset cache area can be determined based on the capacity of the preset cache area.

[0169] After determining the number of items to be moved, the row and column information of each submatrix to be updated and the first row and column information of multiple first submatrixes can be matched. As mentioned above, the matching can be done through the mapping relationship between the row and column information to determine the first submatrix that matches each submatrix to be updated. These first submatrixes are the first submatrixes that need to be moved at the moment, and the number of first submatrixes is the same as the number of items to be moved.

[0170] Then, the first submatrix that needs to be moved is moved from global memory to the preset cache area of ​​the computation unit corresponding to each submatrix to be updated. When subsequent computation units retrieve the first submatrix for computation, they can retrieve the target first submatrix that requires general matrix multiplication from the preset cache area, thereby reducing the number of times the target first submatrix that requires general matrix multiplication is retrieved from global memory. This can improve the computational efficiency of the computation unit.

[0171] Referring to Figure 5, assuming the preset cache can only cache three first submatrices at a time, the number of data transfers is 3. The submatrix to be updated is the one in the first row and first column. Therefore, the corresponding first submatrix is ​​the first submatrix in the first row of the first matrix. This first submatrix has three groups. We can first transfer the three first submatrices in the first row of group A11 from global memory to the preset cache. After the first submatrix in the preset cache is transferred and read, we then transfer the first submatrix in the first row of group A12 to the preset cache before starting the next round of general matrix multiplication.

[0172] It should be noted that, assuming there are three computing units, the submatrix to be updated in the first row of group C1 can be assigned to the first computing unit, the submatrix to be updated in the second row of group C1 to the second computing unit, and the submatrix to be updated in the third row of group C1 to the third computing unit. This achieves load balancing among multiple computing units, further improving computational efficiency.

[0173] After all the submatrices to be updated in group C1 have been updated, the submatrices to be updated in the first row of group C2 can be assigned to the first computation unit, the submatrices to be updated in the second row of group C2 can be assigned to the second computation unit, and the submatrices to be updated in the third row of group C2 can be assigned to the third computation unit. In this way, the submatrices to be updated in each group can be updated sequentially.

[0174] In step 130, the second submatrix corresponding to each submatrix to be updated is determined in the second matrix, and the second submatrix is ​​moved from global memory to the intermediate cache area.

[0175] As can be seen from the general matrix multiplication operation described above, updating the matrix to be updated also requires a second matrix. For each submatrix to be updated, the update can be achieved through the first and second submatrices of each submatrix.

[0176] The second matrix can be partitioned into blocks to obtain the second submatrix corresponding to the submatrix to be updated within the first matrix. Then, the second submatrix is ​​moved from global memory to a transfer buffer. Subsequent computation units can retrieve the second submatrix from the transfer buffer when performing general matrix multiplication operations, thus reducing the number of interactions with global memory. Compared to global memory, the transfer buffer offers higher transfer speeds and lower latency; therefore, pre-storing the second submatrix in the transfer buffer allows computation units to quickly retrieve it during matrix multiplication operations, thereby improving computational efficiency.

[0177] In some implementations, determining the second submatrix corresponding to each submatrix to be updated in the second matrix includes:

[0178] (1.1) Determine the second threshold size of the matrix that the second buffer of each computing unit can store. The second buffer is used to retrieve and store the target second submatrix that requires general matrix multiplication operation corresponding to each submatrix to be updated from the transit buffer.

[0179] (1.2) Divide the second matrix into blocks according to the second threshold size to obtain multiple second sub-matrices;

[0180] (1.3) Determine the second submatrix corresponding to each submatrix to be updated among multiple second submatrixes.

[0181] Each computation unit also includes a second buffer. The second buffer is used to retrieve and store the target second submatrix that needs to be performed on the general matrix multiplication operation for each submatrix to be updated from the preset buffer. In other words, the second submatrix that the computation unit is about to perform the general matrix multiplication operation on is moved from the intermediate buffer to the second buffer in advance. The matrix computation unit in the subsequent computation unit can directly retrieve the target second submatrix that needs to be performed on the general matrix multiplication operation from the second buffer, which can further improve the computation efficiency of the computation unit.

[0182] However, the memory of the second buffer is limited, so a second threshold size needs to be determined for the matrices that the second buffer can store. The second threshold size is the largest matrix size that the second buffer can cache. Therefore, the size of the second submatrix moved to the second buffer must be smaller than this second threshold size.

[0183] In this application, a dual storage area is set for the second buffer. The second submatrix in one storage area can be read and calculated by the matrix calculation unit, while the second submatrix in the other storage area can wait for the next read and calculation by the matrix calculation unit. After the second submatrix in one of the dual storage areas is read and calculated, the data in that storage area can be deleted, and then a new second submatrix can be obtained from the preset buffer and stored in that storage area.

[0184] Therefore, in this application, the second matrix can be divided into blocks according to the second threshold size to obtain multiple second sub-matrices, so that the second sub-matrices meet the dual storage area requirements of the first buffer, such as the sum of the sizes of the two second sub-matrices being less than the second threshold size.

[0185] Finally, the target row and column information of each submatrix to be updated within the matrix to be updated is determined, as well as the second row and column information of each of the multiple second submatrices within the second matrix. Based on the second row and column information, the second submatrix that matches the target row and column information of each submatrix to be updated is identified from among the multiple second submatrices. For example, referring to Figure 5, assuming the target row and column information of the submatrix to be updated is the first row and first column of the matrix to be updated, then all second submatrices in the first column of the second matrix are identified as the second submatrices corresponding to the matrix to be updated. Then, the second submatrices are moved from global memory to a transit cache.

[0186] In step 140, the first target submatrix requiring general matrix multiplication is obtained from the preset buffer for each submatrix to be updated, and the second target submatrix requiring general matrix multiplication is obtained from the transit buffer for each submatrix to be updated.

[0187] In this application, the first submatrix corresponding to the submatrix to be updated is pre-stored in a preset cache area, and the second submatrix corresponding to the submatrix to be updated is pre-stored in a transit cache area. When the subsequent computing unit performs calculations, it can obtain the target first submatrix that requires general matrix multiplication operation for each submatrix to be updated from the preset cache area, and obtain the target second submatrix that requires general matrix multiplication operation for each submatrix to be updated from the transit cache area.

[0188] In some implementations, obtaining the target first submatrix requiring general matrix multiplication for each submatrix to be updated from a preset buffer, and obtaining the target second submatrix requiring general matrix multiplication for each submatrix to be updated from a transit buffer, includes:

[0189] (1.1) Determine the candidate first submatrix corresponding to each submatrix to be updated in the preset buffer;

[0190] (1.2) Move the candidate first submatrix from the preset buffer to the first buffer of the computing unit corresponding to each submatrix to be updated, and obtain the target first submatrix that requires general matrix multiplication operation corresponding to each submatrix to be updated from the candidate first submatrix in the first buffer;

[0191] (1.3) Determine the candidate second submatrix corresponding to each submatrix to be updated in the transit buffer;

[0192] (1.4) Move the candidate second submatrix to the second buffer of the computation unit corresponding to each submatrix to be updated, and obtain the target second submatrix that requires general matrix multiplication operation for each submatrix to be updated from the candidate second submatrix in the second buffer.

[0193] In the preset buffer, there are multiple first sub-matrices. These multiple first sub-matrices can be sorted to obtain a sorting result. Then, based on the sorting result, the first sub-matrices to be used by the matrix calculation unit are determined, and these first sub-matrices are then determined as candidate first sub-matrices.

[0194] The candidate first submatrix is ​​moved from the preset buffer to the first buffer of the computation unit corresponding to each submatrix to be updated. There can be multiple candidate first submatrixes in the first buffer. From the multiple candidate first submatrixes, the target first submatrix corresponding to each submatrix to be updated that requires general matrix multiplication can be selected according to the sorting result.

[0195] There are multiple second submatrices in the transit buffer. These multiple second submatrices can be sorted to obtain a sorting result. Then, based on the sorting result, the second submatrices to be used by the matrix calculation unit are determined. These second submatrices are then identified as candidate second submatrices.

[0196] The candidate first submatrix is ​​moved from the preset buffer to the second buffer of the computation unit corresponding to each submatrix to be updated. In the second buffer, there can be multiple candidate second submatrixes. From the multiple candidate second submatrixes, the target second submatrix corresponding to each submatrix to be updated that requires general matrix multiplication can be selected according to the sorting result.

[0197] Referring to Figure 8, which is a schematic diagram of the structure of the first buffer and the second buffer provided in an embodiment of this application, the first buffer includes a first sub-region and a second sub-region, and the second buffer includes a third sub-region and a fourth sub-region. Either the first or second sub-region can be used to store the first sub-matrix. Either the third or fourth sub-region can be used to store the second sub-matrix.

[0198] In some implementations, moving the candidate first submatrix from a preset cache to the first buffer of the computation unit corresponding to each submatrix to be updated includes:

[0199] (1.2.1) Determine the first free sub-region from the first sub-region and the second sub-region;

[0200] (1.2.2) Move the candidate first submatrix from the preset buffer to the first free subregion.

[0201] In the first sub-region and the second sub-region, if either sub-region does not contain matrix data, then that sub-region can be designated as the first free sub-region, and then the candidate first sub-matrix can be moved from the preset cache to the first free sub-region.

[0202] When the matrix calculation unit reads the target first submatrix that requires general matrix multiplication from the first subregion or the second subregion, the subregion occupied by the target first submatrix cannot transmit the candidate first submatrix.

[0203] After the matrix calculation unit finishes reading and calculating the target first submatrix, it can delete data from the sub-region occupied by the target first submatrix. For example, if the occupied sub-region is the first sub-region, the target first submatrix can be deleted from the first sub-region, and the first sub-region can be updated to the first free sub-region for storing other candidate first submatrixes. The matrix calculation unit can then read and use the candidate first submatrixes in the second sub-region.

[0204] In some implementations, moving the candidate second submatrix to the second buffer of the computation unit corresponding to each submatrix to be updated includes:

[0205] (1.4.1) Determine the second free sub-region within the third and fourth sub-regions;

[0206] (1.4.2) Move the candidate second submatrix from the preset buffer to the second free subregion.

[0207] In the third and fourth sub-regions, if either sub-region has no matrix data, it can be designated as the second free sub-region, and then the candidate second sub-matrix can be moved from the preset buffer to the second free sub-region.

[0208] When the matrix computation unit reads the target second submatrix that requires general matrix multiplication from the third or fourth sub-region, the sub-region occupied by the target second submatrix cannot transmit the candidate second submatrix.

[0209] After the matrix calculation unit finishes reading and calculating the target second submatrix, it can delete data from the sub-region occupied by the target second submatrix. For example, if the occupied sub-region is the third sub-region, the target second submatrix can be deleted from the third sub-region, and the third sub-region can be updated to the second free sub-region for storing other candidate second submatrixes. The matrix calculation unit can then read and use the candidate second submatrixes within the fourth sub-region.

[0210] As can be seen from the above, in this application, a dual storage area is set for the first buffer and the second buffer of the computing unit. This allows the first buffer and the second buffer to continuously update and store the submatrices to be calculated later, thereby improving the computing efficiency of the computing unit.

[0211] In step 150, a general matrix multiplication operation is performed on each submatrix to be updated, the target first submatrix and the target second submatrix corresponding to each submatrix to be updated according to the computing unit corresponding to each submatrix to be updated, to obtain the updated submatrix corresponding to each submatrix to be updated, and an updated matrix corresponding to the submatrix to be updated is generated according to each updated submatrix.

[0212] In this application, the calculation formula for the general matrix by the computing unit is: C←αAB+βC. Where A is the first matrix, B is the second matrix, C on the right side of the formula is the matrix to be updated, C on the left side of the formula is the updated matrix, α is a first preset floating-point number, and β is a second preset floating-point number.

[0213] Therefore, when updating a submatrix to be updated, we can first determine the first and second submatrices corresponding to the submatrix to be updated. Then, we multiply the first preset floating-point number, the first submatrix, and the second submatrix to obtain the first multiplication result, which can be understood as the target submatrix. Next, we multiply the second preset floating-point number and the submatrix to be updated to obtain the second multiplication result. Finally, we add the first and second multiplication results to obtain the updated submatrix corresponding to the submatrix to be updated. The update matrix corresponding to the submatrix to be updated can be generated from each updated submatrix; for example, each updated submatrix can be updated to the corresponding row and column position of the submatrix to be updated, thus generating the update matrix corresponding to the submatrix to be updated.

[0214] In some implementations, a general matrix multiplication operation is performed on each submatrix to be updated, the target first submatrix, and the target second submatrix corresponding to each submatrix to be updated, based on the computational unit corresponding to each submatrix to be updated, to obtain the updated submatrix corresponding to each submatrix to be updated, including:

[0215] (1.1) Input the first target submatrix and the second target submatrix into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated to perform matrix multiplication operation to obtain the target submatrix corresponding to each submatrix to be updated;

[0216] (1.2) Input each submatrix to be updated and the target submatrix into the vector calculation unit of the calculation unit corresponding to each submatrix to be updated to perform vector addition operation to obtain the updated submatrix corresponding to each submatrix to be updated.

[0217] As can be seen from the above, the calculation unit includes a matrix calculation unit and a vector calculation unit. The first target submatrix and the second target submatrix can be input into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated to perform matrix multiplication operations, thereby obtaining the target submatrix corresponding to each submatrix to be updated.

[0218] Then, each submatrix to be updated and the target submatrix are input into the vector calculation unit of the calculation unit corresponding to each submatrix to be updated for vector addition operation to obtain the updated submatrix corresponding to each submatrix to be updated.

[0219] Matrix computation units are better suited for matrix multiplication, while vector computation units are better suited for vector addition. By assigning different computation steps to different units, the computational efficiency of the computation units can be improved.

[0220] In some implementations, after inputting the first target submatrix and the second target submatrix into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated for matrix multiplication to obtain the target submatrix corresponding to each submatrix to be updated, the method further includes:

[0221] (2.1) Delete the first sub-matrix of the target from the first buffer to update the first free sub-region in the first buffer;

[0222] (2.2) Remove the target second submatrix from the second buffer to update the second free subregion in the second buffer.

[0223] Understandably, after inputting the target first submatrix and target second submatrix into the matrix calculation unit corresponding to each submatrix to be updated for matrix multiplication to obtain the target submatrix corresponding to each submatrix to be updated, the matrix calculation unit has completed the calculation of the target first submatrix and target second submatrix. Therefore, the target first submatrix can be deleted from the first buffer to update the first free subregion in the first buffer. For example, if the target first submatrix occupies the first subregion, it can be deleted from the first subregion, thereby updating the first subregion to the first free subregion. Similarly, if the target second submatrix occupies the third subregion, it can be deleted from the third subregion, thereby updating the third subregion to the second free subregion.

[0224] The advantage of this approach is that it allows for continuous updating of the free sub-regions in the first and second buffers. This enables the input of candidate first submatrices for subsequent matrix multiplication operations into the first buffer, and candidate second submatrices for subsequent matrix multiplication operations into the second buffer. This improves the data transmission efficiency of the computing unit, thereby enhancing its computational efficiency.

[0225] It should be noted that in this application, by dividing the matrix to be updated, the first matrix, and the second matrix into blocks of reasonable size, the first buffer of each computing unit can store the first sub-matrix that meets its first threshold size, and the second buffer of each computing unit can store the second sub-matrix that meets its second threshold size. This can improve the data transmission and storage efficiency, thereby enabling the matrix computing unit to process different sub-matrices programmatically, so as to improve the computing efficiency of the entire computing unit.

[0226] Secondly, by evenly distributing multiple matrices to be updated to each computing unit, load balancing can be achieved, preventing waste of computing power and improving the computing efficiency of the computing unit.

[0227] Furthermore, before performing general matrix multiplication in each computing unit, the number of interactions between the computing unit and global memory can be reduced by pre-storing the first submatrix to a preset cache area and the second submatrix to a transfer cache area, thereby improving the computing efficiency of the computing unit.

[0228] In addition, by setting up dual storage areas in the first and second buffers of the computing unit, the matrix data to be calculated can be continuously transferred to the first and second buffers, which can improve the computing efficiency of the computing unit.

[0229] In this application, the above-mentioned method improves the computational efficiency of each computing unit, that is, it improves the computational efficiency of the computing unit when performing general matrix multiplication, thereby improving the computational efficiency of the neural network processor when performing general matrix multiplication and making full use of the computing power of the neural network processor.

[0230] As described above, in this embodiment, the neural network processor includes global memory, a transit cache, and multiple computing units. It obtains multiple sub-matrices to be updated by dividing the matrix to be updated into blocks, and evenly distributes these sub-matrices to each computing unit. A first sub-matrix corresponding to each sub-matrix to be updated is determined in the first matrix, and the first sub-matrix is ​​moved from global memory to a preset cache of the computing unit corresponding to each sub-matrix to be updated. A second sub-matrix corresponding to each sub-matrix to be updated is determined in the second matrix, and the second sub-matrix is ​​moved from global memory to the transit cache. A target first sub-matrix requiring general matrix multiplication is obtained from the preset cache, and a target second sub-matrix requiring general matrix multiplication is obtained from the transit cache. General matrix multiplication is performed on each sub-matrix to be updated, the target first sub-matrix, and the target second sub-matrix corresponding to each sub-matrix to be updated by the computing unit corresponding to each sub-matrix to be updated, to obtain an updated sub-matrix corresponding to each sub-matrix to be updated, and an updated matrix corresponding to the sub-matrix to be updated is generated based on each updated sub-matrix.

[0231] Therefore, by dividing the matrix to be updated into blocks, multiple sub-matrices to be updated are obtained. The first matrix and the second matrix are also divided into blocks to obtain the first sub-matrix and the second sub-matrix corresponding to each sub-matrix to be updated. The first sub-matrix and the second sub-matrix to be processed by each computing unit are pre-moved from global memory to the corresponding cache area. This reduces the number of times data is moved from global memory when each computing unit performs general matrix multiplication. Furthermore, the balanced distribution of the sub-matrix to be updated, the first sub-matrix, and the second sub-matrix processed by each computing unit enables load balancing when multiple computing units perform general matrix multiplication, thereby improving the efficiency of each computing unit in performing general matrix multiplication and thus efficiently utilizing the computing power of the neural network processor.

[0232] Please refer to Figure 9, which is another schematic flowchart of the data processing method provided in an embodiment of this application. The data processing method may include the following steps:

[0233] Step 201: Divide the matrix to be updated into blocks to obtain multiple sub-matrices to be updated, and distribute the multiple sub-matrices to be updated evenly to each computing unit;

[0234] Step 202: Determine the first threshold size of the matrix that the first buffer of each computing unit can store. The first buffer is used to retrieve and store the target first submatrix that requires general matrix multiplication operation corresponding to each submatrix to be updated from the preset buffer.

[0235] Step 203: Divide the first matrix into blocks according to the first threshold size to obtain multiple first sub-matrices;

[0236] Step 204: Determine the first submatrix corresponding to each submatrix to be updated among multiple first submatrixes;

[0237] Step 205: Determine the second threshold size of the matrix that the second buffer of each computing unit can store. The second buffer is used to retrieve and store the target second submatrix that requires general matrix multiplication operation corresponding to each submatrix to be updated from the transit buffer.

[0238] Step 206: Divide the second matrix into blocks according to the second threshold size to obtain multiple second sub-matrices;

[0239] Step 207: Determine the second submatrix corresponding to each submatrix to be updated among multiple second submatrixes;

[0240] Step 208: Determine the candidate first submatrix corresponding to each submatrix to be updated in the preset buffer;

[0241] Step 209: Move the candidate first submatrix from the preset buffer to the first buffer of the computing unit corresponding to each submatrix to be updated, and obtain the target first submatrix that requires general matrix multiplication operation for each submatrix to be updated from the candidate first submatrix in the first buffer.

[0242] Step 210: Determine the candidate second submatrix corresponding to each submatrix to be updated in the transit buffer;

[0243] Step 211: Move the candidate second submatrix to the second buffer of the computation unit corresponding to each submatrix to be updated, and obtain the target second submatrix that requires general matrix multiplication operation for each submatrix to be updated from the candidate second submatrix in the second buffer;

[0244] Step 212: Input the first target submatrix and the second target submatrix into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated to perform matrix multiplication operations, and obtain the target submatrix corresponding to each submatrix to be updated;

[0245] Step 213: Input each submatrix to be updated and the target submatrix into the vector calculation unit of the calculation unit corresponding to each submatrix to be updated to perform vector addition operation to obtain the updated submatrix corresponding to each submatrix to be updated, and generate the updated matrix corresponding to the submatrix to be updated based on each updated submatrix.

[0246] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed description of the data processing method above, which will not be repeated here.

[0247] Please refer to Figure 10, which is a schematic diagram illustrating the structure of a data processing apparatus provided in this application. This data processing apparatus can perform the data processing methods described in the above embodiments.

[0248] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0249] As shown in Figure 10, the data processing device 300 is applied to a neural network processor. The neural network processor includes global memory, a transit cache, and multiple computing units. The data processing device 300 includes:

[0250] Blocking module 310 is used to divide the matrix to be updated into multiple sub-matrices to be updated, and to distribute the multiple sub-matrices to be updated evenly to each computing unit.

[0251] The first determining module 320 is used to determine the first submatrix corresponding to each submatrix to be updated in the first matrix, and to move the first submatrix from global memory to the preset cache area of ​​the computing unit corresponding to each submatrix to be updated;

[0252] The second determining module 330 is used to determine the second submatrix corresponding to each submatrix to be updated in the second matrix, and move the second submatrix from global memory to the intermediate cache area;

[0253] The acquisition module 340 is used to acquire the target first submatrix that requires general matrix multiplication operation for each submatrix to be updated from the preset buffer, and to acquire the target second submatrix that requires general matrix multiplication operation for each submatrix to be updated from the transit buffer.

[0254] The calculation module 350 is used to perform a general matrix multiplication operation on each submatrix to be updated, the target first submatrix and the target second submatrix corresponding to each submatrix to be updated according to the calculation unit corresponding to each submatrix to be updated, to obtain the updated submatrix corresponding to each submatrix to be updated, and to generate the updated matrix corresponding to the submatrix to be updated based on each updated submatrix.

[0255] In some implementations, the first determining module 320 is configured to:

[0256] Obtain the first matrix, and divide the first matrix into blocks to obtain the first target matrix corresponding to each submatrix to be updated;

[0257] The first target matrix is ​​grouped to obtain the first submatrix corresponding to each submatrix to be updated;

[0258] Each first submatrix is ​​moved from global memory to the preset cache area in the corresponding computation unit.

[0259] In some implementations, the first determining module 320 includes a first determining submodule, a block submodule, and a second determining submodule;

[0260] The first determining submodule is used to determine the first threshold size of the matrix that the first buffer of each computing unit can store. The first buffer is used to obtain and store the target first submatrix that requires general matrix multiplication operation corresponding to each submatrix to be updated from the preset buffer.

[0261] The block division submodule is used to divide the first matrix into blocks according to the first threshold size to obtain multiple first sub-matrices;

[0262] The second determination submodule is used to determine the first submatrix corresponding to each submatrix to be updated among multiple first submatrixes.

[0263] In some implementations, the second determining submodule is used for:

[0264] Determine a first threshold size for the matrix that the first buffer of each computation unit can store. The first buffer is used to retrieve and store the target first submatrix that requires general matrix multiplication operation for each submatrix to be updated from a preset buffer.

[0265] The first matrix is ​​divided into blocks according to the first threshold size to obtain multiple first sub-matrices;

[0266] The first submatrix corresponding to each submatrix to be updated is determined from multiple first submatrixes.

[0267] In some implementations, the first determining module 320 is configured to:

[0268] Determine the capacity of the preset buffer area for each computational unit corresponding to the submatrix to be updated;

[0269] Based on the capacity and the first row and column information, determine the first submatrix that needs to be moved from the first submatrix that matches the row and column of each submatrix to be updated;

[0270] Move the first submatrix that needs to be moved from global memory to the preset cache area of ​​the computation unit corresponding to each submatrix to be updated.

[0271] In some implementations, the second determining module 330 is used for:

[0272] Determine a second threshold size for the matrix that the second buffer of each computation unit can store. The second buffer is used to retrieve and store the target second submatrix that requires general matrix multiplication operation for each submatrix to be updated from the transit buffer.

[0273] The second matrix is ​​divided into blocks according to the second threshold size to obtain multiple second sub-matrices;

[0274] The second submatrix corresponding to each submatrix to be updated is determined from multiple second submatrixes.

[0275] In some implementations, the acquisition module 340 is used for:

[0276] Determine the candidate first submatrix corresponding to each submatrix to be updated in the preset cache area;

[0277] The candidate first submatrix is ​​moved from the preset buffer to the first buffer of the computation unit corresponding to each submatrix to be updated. The target first submatrix that needs to be universally multiplied is obtained from the candidate first submatrix in the first buffer.

[0278] In the transit buffer, determine the candidate second submatrix corresponding to each submatrix to be updated;

[0279] The candidate second submatrix is ​​moved to the second buffer of the computation unit corresponding to each submatrix to be updated, and the target second submatrix that requires general matrix multiplication is obtained from the candidate second submatrix in the second buffer.

[0280] In some implementations, the first buffer includes a first sub-region and a second sub-region, the second buffer includes a third sub-region and a fourth sub-region, and the first determining module 320 is used for:

[0281] The first free sub-region is determined within the first sub-region and the second sub-region;

[0282] Move the candidate first sub-matrix from the preset cache to the first free sub-region;

[0283] The second determining module 330 is used for:

[0284] The second free sub-region is determined in the third and fourth sub-regions;

[0285] Move the candidate second sub-matrix from the preset cache to the second free sub-region.

[0286] In some implementations, the computing module 350 is used for:

[0287] The first and second target submatrices are input into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated to perform matrix multiplication, thereby obtaining the target submatrix corresponding to each submatrix to be updated;

[0288] Each submatrix to be updated and the target submatrix are input into the vector computation unit of the computation unit corresponding to each submatrix to be updated for vector addition operation to obtain the updated submatrix corresponding to each submatrix to be updated.

[0289] In some implementations, after inputting the first target submatrix and the second target submatrix into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated for matrix multiplication to obtain the target submatrix corresponding to each submatrix to be updated, the calculation module 350 is used for:

[0290] Remove the first sub-matrix of the target from the first buffer to update the first free sub-region in the first buffer;

[0291] Remove the target second submatrix from the second buffer to update the second free subregion in the second buffer.

[0292] In some implementations, the block module 310 is used for:

[0293] Determine the total number of multiple computing units;

[0294] The multiple matrices to be updated are evenly distributed to each computing unit based on the total number.

[0295] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed description of the data processing method above, which will not be repeated here.

[0296] In this embodiment, the neural network processor includes global memory, a transit cache, and multiple computing units. A block-splitting module 310 is used to divide the matrix to be updated into blocks to obtain multiple sub-matrices to be updated, and to evenly distribute these sub-matrices to each computing unit. A first determining module 320 is used to determine the first sub-matrix corresponding to each sub-matrix to be updated in the first matrix, and to move the first sub-matrix from global memory to a preset cache of the computing unit corresponding to each sub-matrix to be updated. A second determining module 330 is used to determine the second sub-matrix corresponding to each sub-matrix to be updated in the second matrix, and to move the second sub-matrix from global memory to a preset cache of the computing unit corresponding to each sub-matrix to be updated. The data is transferred from storage to a transit buffer; the acquisition module 340 is used to acquire the target first submatrix that needs to be multiplied by general matrix operation for each submatrix to be updated from the preset buffer, and to acquire the target second submatrix that needs to be multiplied by general matrix operation for each submatrix to be updated from the transit buffer; the calculation module 350 is used to perform general matrix multiplication on each submatrix to be updated, the target first submatrix and the target second submatrix corresponding to each submatrix to be updated according to the calculation unit corresponding to each submatrix to be updated, to obtain the updated submatrix corresponding to each submatrix to be updated, and to generate the updated matrix corresponding to the submatrix to be updated according to each updated submatrix.

[0297] Therefore, by dividing the matrix to be updated into blocks, multiple sub-matrices to be updated are obtained. The first matrix and the second matrix are also divided into blocks to obtain the first sub-matrix and the second sub-matrix corresponding to each sub-matrix to be updated. The first sub-matrix and the second sub-matrix to be processed by each computing unit are pre-moved from global memory to the corresponding cache area. This reduces the number of times data is moved from global memory when each computing unit performs general matrix multiplication. Furthermore, the balanced distribution of the sub-matrix to be updated, the first sub-matrix, and the second sub-matrix processed by each computing unit enables load balancing when multiple computing units perform general matrix multiplication, thereby improving the efficiency of each computing unit in performing general matrix multiplication and thus efficiently utilizing the computing power of the neural network processor.

[0298] Please refer to Figure 11, which is a schematic diagram of the structure of a computer device provided in an embodiment of this application. This application also provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described data processing method. This computer device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0299] Please refer to Figure 11, which illustrates the hardware structure of a computer device according to another embodiment. The computer device includes:

[0300] The processor 401 may specifically be a neural network processor, used to execute relevant programs to implement the technical solutions provided in the embodiments of this application;

[0301] The memory 402 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 402 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 402 and is called and executed by the processor 401 using the data processing method of the embodiments of this application.

[0302] Input / output interface 403 is used to implement information input and output;

[0303] The communication interface 404 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0304] Bus 405 transmits information between various components of the device (e.g., processor 401, memory 402, input / output interface 403, and communication interface 404);

[0305] The processor 401, memory 402, input / output interface 403 and communication interface 404 are connected to each other within the device via bus 405.

[0306] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described data processing method.

[0307] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0308] The data processing method, apparatus, computer equipment, and storage medium provided in this application embodiment obtain multiple sub-matrices to be updated by dividing the matrix to be updated into blocks, and evenly distributing the multiple sub-matrices to be updated to each computing unit; determining the first sub-matrix corresponding to each sub-matrix to be updated in the first matrix, and moving the first sub-matrix from global memory to the preset cache area of ​​the computing unit corresponding to each sub-matrix to be updated; determining the second sub-matrix corresponding to each sub-matrix to be updated in the second matrix, and moving the second sub-matrix from global memory to the intermediate cache area; obtaining the target first sub-matrix requiring general matrix multiplication operation corresponding to each sub-matrix to be updated from the preset cache area, and obtaining the target second sub-matrix requiring general matrix multiplication operation corresponding to each sub-matrix to be updated from the intermediate cache area; performing general matrix multiplication operation on each sub-matrix to be updated, the target first sub-matrix, and the target second sub-matrix corresponding to each sub-matrix to be updated according to the computing unit corresponding to each sub-matrix to be updated, to obtain the updated sub-matrix corresponding to each sub-matrix to be updated, and generating the updated matrix corresponding to the sub-matrix to be updated based on each updated sub-matrix.

[0309] Therefore, by dividing the matrix to be updated into blocks, multiple sub-matrices to be updated are obtained. The first matrix and the second matrix are also divided into blocks to obtain the first sub-matrix and the second sub-matrix corresponding to each sub-matrix to be updated. The first sub-matrix and the second sub-matrix to be processed by each computing unit are pre-moved from global memory to the corresponding cache area. This reduces the number of times data is moved from global memory when each computing unit performs general matrix multiplication. Furthermore, the balanced distribution of the sub-matrix to be updated, the first sub-matrix, and the second sub-matrix processed by each computing unit enables load balancing when multiple computing units perform general matrix multiplication, thereby improving the efficiency of each computing unit in performing general matrix multiplication and thus efficiently utilizing the computing power of the neural network processor.

[0310] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0311] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0312] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0313] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0314] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0315] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0316] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0317] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0318] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0319] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0320] The data processing method, apparatus, computer equipment, and storage medium provided in the embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and its core ideas. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope, which do not limit the scope of rights of the embodiments of this application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of this application should be within the scope of rights of the embodiments of this application.

Claims

1. A data processing method, characterized in that, Applied to a neural network processor, the neural network processor including global memory, a transit cache, and multiple computing units, the method includes: The matrix to be updated is divided into blocks to obtain multiple sub-matrices to be updated, and these multiple sub-matrices to be updated are evenly distributed to each computing unit. In the first matrix, the first submatrix corresponding to each submatrix to be updated is determined, and the first submatrix is ​​moved from the global memory to the preset cache area of ​​the computing unit corresponding to each submatrix to be updated; In the second matrix, determine the second submatrix corresponding to each submatrix to be updated, and move the second submatrix from the global memory to the transit cache area; Obtain the first target submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated from the preset cache area, and obtain the second target submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated from the transit cache area; Perform a general matrix multiplication operation on each submatrix to be updated, the target first submatrix and the target second submatrix corresponding to each submatrix to be updated using the computing unit corresponding to each submatrix to be updated, to obtain the updated submatrix corresponding to each submatrix to be updated, and generate the updated matrix corresponding to the submatrix to be updated based on each updated submatrix.

2. The data processing method according to claim 1, characterized in that, The process of determining the first submatrix corresponding to each submatrix to be updated in the first matrix includes: A first threshold size of the matrix that can be stored in the first buffer of each computing unit is determined. The first buffer is used to obtain and store the target first submatrix that requires general matrix multiplication operation corresponding to each submatrix to be updated from the preset buffer. The first matrix is ​​divided into blocks according to the first threshold size to obtain multiple first sub-matrices; The first submatrix corresponding to each submatrix to be updated is determined from the plurality of first submatrixes.

3. The data processing method according to claim 2, characterized in that, The step of determining the first submatrix corresponding to each submatrix to be updated among the plurality of first submatrixes includes: Determine the target row and column information of each submatrix to be updated in the matrix to be updated, and determine the first row and column information of each of the plurality of first submatrices in the first matrix; Based on the first row and column information, a first submatrix that matches the target row and column information of each submatrix to be updated is determined from among the plurality of first submatrixes.

4. The data processing method according to claim 3, characterized in that, The step of moving the first submatrix from the global memory to the preset cache area of ​​the computing unit corresponding to each submatrix to be updated includes: Determine the capacity of the preset cache area for each computational unit corresponding to the submatrix to be updated; Based on the capacity and the first row and column information, the first submatrix that needs to be moved is determined from the first submatrix that matches the row and column of each submatrix to be updated; The first submatrix that needs to be moved is moved from the global memory to the preset cache area of ​​the computing unit corresponding to each submatrix to be updated.

5. The data processing method according to claim 1, characterized in that, Determining the second submatrix corresponding to each submatrix to be updated in the second matrix includes: A second threshold size of the matrix that can be stored in the second buffer of each computing unit is determined. The second buffer is used to obtain and store the target second submatrix that requires general matrix multiplication operation corresponding to each submatrix to be updated from the transit buffer. The second matrix is ​​divided into blocks according to the second threshold size to obtain multiple second sub-matrices; The second submatrix corresponding to each submatrix to be updated is determined from the plurality of second submatrixes.

6. The data processing method according to claim 1, characterized in that, The step of obtaining the target first submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated from the preset buffer, and obtaining the target second submatrix requiring general matrix multiplication operation corresponding to each submatrix to be updated from the transit buffer, includes: Determine the candidate first submatrix corresponding to each submatrix to be updated in the preset cache area; The candidate first submatrix is ​​moved from the preset cache to the first buffer of the computing unit corresponding to each submatrix to be updated. In the first buffer, the target first submatrix corresponding to each submatrix to be updated that requires general matrix multiplication is obtained from the candidate first submatrix; In the transit buffer, determine the candidate second submatrix corresponding to each submatrix to be updated; The candidate second submatrix is ​​moved to the second buffer of the computation unit corresponding to each submatrix to be updated, and the target second submatrix requiring general matrix multiplication operation is obtained from the candidate second submatrix in the second buffer.

7. The data processing method according to claim 6, characterized in that, The first buffer includes a first sub-region and a second sub-region, the second buffer includes a third sub-region and a fourth sub-region, and the step of moving the candidate first sub-matrix from the preset buffer to the first buffer of the computation unit corresponding to each sub-matrix to be updated includes: A first free sub-region is determined within the first sub-region and the second sub-region; The candidate first sub-matrix is ​​moved from the preset cache area to the first free sub-region; The step of moving the candidate second submatrix to the second buffer of the computation unit corresponding to each submatrix to be updated includes: A second free sub-region is determined in the third sub-region and the fourth sub-region; The candidate second sub-matrix is ​​moved from the preset cache area to the second free sub-region.

8. The data processing method according to claim 6, characterized in that, The step of performing a general matrix multiplication operation on each submatrix to be updated, the target first submatrix, and the target second submatrix corresponding to each submatrix to be updated using the computing unit corresponding to each submatrix to be updated, to obtain the updated submatrix corresponding to each submatrix to be updated, includes: The first target submatrix and the second target submatrix are input into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated to perform matrix multiplication, thereby obtaining the target submatrix corresponding to each submatrix to be updated; Each submatrix to be updated and the target submatrix are input into the vector calculation unit of the calculation unit corresponding to each submatrix to be updated for vector addition operation to obtain the updated submatrix corresponding to each submatrix to be updated.

9. The data processing method according to claim 8, characterized in that, After inputting the target first submatrix and the target second submatrix into the matrix calculation unit of the calculation unit corresponding to each submatrix to be updated for matrix multiplication to obtain the target submatrix corresponding to each submatrix to be updated, the method further includes: The target first sub-matrix is ​​deleted from the first buffer to update the first free sub-region in the first buffer; The target second sub-matrix is ​​deleted from the second buffer to update the second free sub-region in the second buffer.

10. The data processing method according to claim 1, characterized in that, The step of evenly distributing the plurality of sub-matrices to be updated to each computing unit includes: Determine the total number of the plurality of computing units; The plurality of matrices to be updated are evenly distributed to each computing unit according to the total number.

11. A data processing apparatus, characterized in that, Applied to a neural network processor, the neural network processor including global memory, a transit cache, and multiple computing units, the device includes: The block division module is used to divide the matrix to be updated into multiple sub-matrices to be updated, and to distribute the multiple sub-matrices to be updated evenly to each computing unit. The first determining module is used to determine the first submatrix corresponding to each submatrix to be updated in the first matrix, and to move the first submatrix from the global memory to the preset cache area of ​​the computing unit corresponding to each submatrix to be updated; The second determining module is used to determine the second submatrix corresponding to each submatrix to be updated in the second matrix, and to move the second submatrix from the global memory to the transit cache area; The acquisition module is used to acquire the target first submatrix that requires general matrix multiplication operation corresponding to each submatrix to be updated from the preset cache area, and to acquire the target second submatrix that requires general matrix multiplication operation corresponding to each submatrix to be updated from the transit cache area. The calculation module is used to perform a general matrix multiplication operation on each submatrix to be updated, the target first submatrix and the target second submatrix corresponding to each submatrix to be updated according to the calculation unit corresponding to each submatrix to be updated, to obtain the updated submatrix corresponding to each submatrix to be updated, and to generate the updated matrix corresponding to the submatrix to be updated according to each updated submatrix.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform the data processing method according to any one of claims 1 to 10.

13. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the data processing method according to any one of claims 1 to 10.

Images