Data processing method and device, electronic equipment and storage medium

Abstract

The invention provides a data processing method, and relates to the technical field of artificial intelligence, in particular to the technical field of chips and the technical field of large models. According to the specific implementation scheme, in response to determining that the data size of a first matrix is larger than the capacity of a first storage space and smaller than or equal to the capacity of a second storage space, the first matrix is carried to the second storage space; a to-be-processed sub-matrix of a to-be-processed matrix is carried to a first storage space, the to-be-processed matrix is determined according to a second matrix, and the data size of the second matrix is larger than or equal to that of the to-be-processed matrix; according to the first matrix and the to-be-processed sub-matrix of the to-be-processed matrix, matrix operation is executed, and a result matrix is obtained. The invention further provides a data processing device, electronic equipment and a storage medium.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the field of chip technology and large model technology. More specifically, the present disclosure provides a data processing method, device, electronic device and storage medium. Background Art

[0002] With the development of artificial intelligence technology, the application of large models is increasing. When using artificial intelligence chips to perform reasoning or training tasks of large models, multiple buffer units of artificial intelligence chips can be used to store the weight matrix and input data of the large model. The capacity of different buffer units is different. As the application scenarios of large models continue to expand, the amount of input data of large models is also increasing, which is often greater than the capacity of the buffer unit corresponding to the input data, resulting in the continuous increase of chip memory resource overhead and the difficulty in effectively improving the utilization of chip resources. Summary of the invention

[0003] The present disclosure provides a data processing method, apparatus, device and storage medium.

[0004] According to one aspect of the present disclosure, a data processing method is provided, the method comprising: in response to determining that the data amount of a first matrix is ​​greater than the capacity of a first storage space and less than or equal to the capacity of a second storage space, moving the first matrix to the second storage space; moving a submatrix to be processed of the matrix to be processed to the first storage space, wherein the matrix to be processed is determined based on the second matrix, and the data amount of the second matrix is ​​greater than or equal to the matrix to be processed; performing matrix operations based on the first matrix and the submatrix to be processed of the matrix to be processed to obtain a result matrix.

[0005] According to another aspect of the present disclosure, a data processing device is provided, which includes: a first buffer unit, a second buffer unit; a data transfer unit, configured to: in response to determining that the data amount of the first matrix is ​​greater than the capacity of the first storage space and less than or equal to the capacity of the second storage space, transfer the first matrix to the second storage space; and transfer the to-be-processed sub-matrix of the to-be-processed matrix to the first storage space, wherein the to-be-processed matrix is ​​determined according to the second matrix, the data amount of the second matrix is ​​greater than or equal to the to-be-processed matrix, the first storage space is located in the first buffer unit, and the second storage space is located in the second buffer unit; a calculation module, configured to: perform matrix operations according to the first matrix and the to-be-processed sub-matrix of the to-be-processed matrix to obtain a result matrix.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising the device provided by the present disclosure.

[0007] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided according to the present disclosure.

[0008] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided. The computer instructions are used to cause a computer to execute the method provided according to the present disclosure.

[0009] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method provided according to the present disclosure is implemented.

[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0012] Figure 1 is a schematic diagram of matrix operations according to an embodiment of the present disclosure;

[0013] Figure 2 is a flow chart of a data processing method according to an embodiment of the present disclosure;

[0014] Figure 3 is a schematic diagram of multiple storage spaces according to an embodiment of the present disclosure;

[0015] Figure 4A is a schematic flow chart of determining an initial submatrix according to an embodiment of the present disclosure;

[0016] Figure 4B is a schematic flow chart of determining a submatrix according to an embodiment of the present disclosure;

[0017] Figure 5 is a schematic diagram of a data processing method according to an embodiment of the present disclosure;

[0018] Figure 6 is a block diagram of a data processing device according to an embodiment of the present disclosure;

[0019] Figure 7 is a block diagram of a data processing apparatus according to an embodiment of the present disclosure; and

[0020] Figure 8is a block diagram of an electronic device to which a data processing method can be applied according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0022] The booming development of artificial intelligence technology has put forward unprecedented demands on the computing power of computing devices. As a result, a large number of artificial intelligence chips such as graphics processing units (GPUs) and artificial intelligence application-specific integrated circuits (AI ASICs) have emerged. For matrix multiplication, which is widely used in the field of deep learning, some artificial intelligence chips are designed with independent storage and computing units to complete its high-performance calculations. For example, considering the differences in storage access characteristics and data reuse of the left matrix, the right matrix, and the output matrix, the artificial intelligence chip can store the data of these three matrices in different on-chip memories to achieve decoupling, so that the three matrices can be accessed independently, thereby improving access efficiency. Overall, compared with traditional central processing units (CPUs), these artificial intelligence chips have achieved huge performance improvements when performing operations such as matrix multiplication, but in some scenarios, the limited capacity of on-chip storage will still lead to repeated memory access, which makes it difficult to further improve the performance of the chip.

[0023] In some embodiments, in order to realize matrix multiplication operation, the left matrix and the right matrix can be divided into blocks based on the number of hardware cores, computing component computing power limit and alignment limit, storage component alignment limit and capacity limit, after fully considering memory access merging, loop unrolling, pipeline parallelism and vector parallelism. And optimize the block strategy including the scale of blocks, calculation loop order, etc. to obtain the optimal block strategy. The optimal block strategy can be obtained through software simulation or hardware measurement, and passed to the kernel function as input, so that the matrix multiplication calculation can make full use of hardware resources in various scenarios to achieve better hardware utilization. The kernel function corresponds to the operation performed by the hardware computing unit.

[0024] For example, for different block strategies, the solution can be obtained according to the matrix multiplication calculation process to determine the optimal block strategy. The size of the left matrix can be m×k. The size of the first submatrix obtained by splitting the left matrix can be block_m×block_k. The first submatrix is ​​loaded into the first buffer space. The size of the right matrix can be k×n. The size of the second submatrix obtained by splitting the right matrix can be block_k×block_n. The second submatrix is ​​loaded into the second buffer space. The computing unit can obtain data from the first buffer space and the second buffer space, and output the result submatrix after completing the calculation. The size of the result submatrix can be block_m×block_n. The result submatrix can be stored in the corresponding buffer space or cache space. The size of the matrix after blocking is limited by the capacity of the first buffer space, the second buffer space and the cache space on the chip. If the amount of data of the left matrix is ​​greater than the capacity of the first buffer space and the amount of data of the right matrix is ​​greater than the capacity of the second buffer space, it is necessary to load data from the global storage unit multiple times, resulting in repeated loading of data. In many deep learning algorithms, the right matrix can be a weight matrix, and the left matrix can be a matrix of input data or intermediate results. The weight matrix is ​​fixed throughout the calculation process, while the input data is changing. A variety of deep learning algorithms can include transformers, convolutional neural networks (CNNs), and fully connected networks (FCNs). For deep learning application scenarios, in order to adapt to the peak computing power of the computing unit in a storage space with limited total capacity, the capacity of the first buffer space on the chip and the capacity of the second buffer space are different. The computing unit can be a systolic array. The following will combine Figure 1 Matrix operations are explained.

[0025] Figure 1 is a schematic diagram of matrix operations according to an embodiment of the present disclosure.

[0026] like Figure 1 As shown, the amount of data of the first matrix A10 may be greater than the capacity of the first storage space bufferA10. The amount of data of the second matrix B10 may be greater than the capacity of the second storage space bufferB10. The first matrix A10 may be split into two first sub-matrices. The two first sub-matrices are respectively the first sub-matrix A100 and the first sub-matrix A101. The second matrix B10 may be split into two second sub-matrices. The two second sub-matrices are respectively the second sub-matrix B100 and the second sub-matrix B101. It can be understood that the first storage space may be the above-mentioned first buffer space. The second storage space may be the above-mentioned second buffer space.

[0027] like Figure 1As shown, according to the first submatrix A100 and the first submatrix A101 and the second submatrix B100 and the second submatrix B101, multiple matrix operations can be performed to obtain an output matrix C10. The output matrix C10 corresponds to multiple third submatrices. The multiple third submatrices include a third submatrix C100, a third submatrix C101, a third submatrix C110, and a third submatrix C111. For example, according to the first submatrix A100 and the second submatrix B100, a matrix operation can be performed to obtain the third submatrix C100. The third submatrix can be stored in the cache space cacheC10.

[0028] like Figure 1 As shown, during multiple matrix operations, the second sub-matrix needs to be loaded repeatedly, resulting in an increase in memory access.

[0029] Based on this, in order to reduce memory access resource overhead, the present disclosure provides a data processing method, which will be described below.

[0030] Figure 2 is a flow chart of a data processing method according to an embodiment of the present disclosure.

[0031] like Figure 2 As shown, the method 200 may include operations S210 to S230.

[0032] In operation S210, in response to determining that the data amount of the first matrix is ​​greater than the capacity of the first storage space and less than or equal to the capacity of the second storage space, the first matrix is ​​moved to the second storage space.

[0033] In the embodiment of the present disclosure, the first storage space may be a storage space of a first buffer unit, and the second storage space may be a storage space of a second buffer unit.

[0034] For example, the first matrix may be the first matrix A10. If the data volume of the first matrix A10 is larger than the capacity of the first storage space bufferA10 and smaller than or equal to the capacity of the second storage space bufferB10, the first matrix A10 may be transferred to the second storage space bufferB10.

[0035] In operation S220, a sub-matrix to be processed of the matrix to be processed is moved to a first storage space.

[0036] In the disclosed embodiment, the matrix to be processed is determined according to the second matrix, and the data volume of the second matrix is ​​greater than or equal to the matrix to be processed. For example, the second matrix can be split to obtain the matrix to be processed. For another example, in scenarios such as large model reasoning or training, the number of parameters of the large model is large, and the corresponding weight matrix has a large data volume. The weight matrix can be used as the above-mentioned second matrix. The second matrix is ​​split so as to be stored in the buffer space of the on-chip storage unit.

[0037] In the embodiment of the present disclosure, the matrix to be processed corresponds to one or more sub-matrices to be processed. For example, the matrix to be processed can be further split so as to store multiple sub-matrices to be processed in the first storage space, thereby reducing the resource overhead required to move the matrix to be processed to the second storage space, and also helping to improve the pipeline execution efficiency of the computing unit. For another example, the matrix to be processed can also be used as a sub-matrix to be processed.

[0038] In operation S230, a matrix operation is performed according to the first matrix and the sub-matrix to be processed of the matrix to be processed to obtain a result matrix.

[0039] In the embodiments of the present disclosure, matrix operations may include various matrix operations such as matrix multiplication, matrix addition, and matrix division.

[0040] Through the embodiments of the present disclosure, according to the amount of data of the matrix, the matrix is ​​stored in the corresponding storage space, and the first matrix is ​​no longer fixedly stored in the first storage space, so that the on-chip storage resources can be effectively utilized. In particular, in scenarios such as large model reasoning or training, the amount of data of the weight matrix is ​​large and it is difficult to move all of it to the second storage space. In this case, moving all of the first matrices with a smaller amount of data to the second storage space can reduce repeated memory accesses to the first matrix, improve memory access efficiency, make full use of the storage resources of the chip's buffer unit, and also help improve computing efficiency.

[0041] It can be understood that the method of the present disclosure is described above, and the method of the present disclosure will be further described below in conjunction with the on-chip storage resources of the chip.

[0042] Figure 3 is a schematic diagram of multiple storage spaces according to an embodiment of the present disclosure.

[0043] like Figure 3 As shown, the plurality of storage spaces include a first storage space bufferA30 and a second storage space bufferB30. The first storage space bufferA30 may be the first storage space bufferA10 described above. The second storage space bufferB30 may be the second storage space bufferB10 described above.

[0044] In some embodiments, in some implementations of the above operation S210, in response to determining that the amount of data of the first matrix is ​​greater than the capacity of the first storage space and less than or equal to the capacity of the second storage space, the first matrix may be moved from the first cache space to the second storage space. The first cache space may be a space in a second-level cache unit (L2 cache). Figure 3 As shown, when the amount of data of the first matrix A30 is greater than the capacity of the first storage space bufferA30 and less than the capacity of the second storage space B30, the first matrix A30 is moved to the second storage space bufferB30. It can be understood that the first matrix A30 in the second storage space bufferB30 can be a transposed matrix. The scale of the first matrix A30 in the second storage space bufferB30 can be k×m.

[0045] In some embodiments, in some implementations of the above operation S220, the sub-matrix to be processed of the matrix to be processed is moved from the second cache space to the first storage space. The first cache space may be a space within a second-level cache unit (L2cache). For example, all sub-matrices to be processed of the matrix B100 to be processed may be moved from the second cache space to bufferA30. The second matrix corresponds to a plurality of matrices to be processed. The plurality of matrices to be processed may include, for example, a matrix to be processed B100, a matrix to be processed B101, and a matrix to be processed B102. It is understood that the size of the matrix to be processed B100 may be block_n×k.

[0046] In some embodiments, in some implementations of the above operation S230, a matrix operation is performed based on the first matrix and the submatrix to be processed of the matrix to be processed to obtain a matrix to be transposed. The matrix to be transposed is transposed to obtain a result matrix. The matrix to be transposed can be stored in the third cache space. The result matrix can be stored in the fourth cache space. Figure 3As shown, according to the first matrix A30 and the multiple sub-matrices to be processed of the matrix B100 to be processed, the matrix to be transposed can be obtained by performing matrix operations. The matrix to be transposed can be transposed to obtain the result matrix C300. The result matrix C300 can be stored in the fourth cache space cacheC30. It can be understood that the first cache space, the second cache space, the third cache space, and the fourth cache space can be cache spaces in different secondary cache units, or can be storage spaces in the same secondary cache unit. Through the embodiment of the present disclosure, the calculation logic of the systolic array is almost not adjusted, which reduces the application threshold of the present disclosure and the cost required to adjust the systolic array. It can be understood that the systolic array can use the matrix of the first storage space as the left matrix and the matrix of the second storage space as the right matrix. When the storage space of the first matrix and the second matrix changes, the result of the matrix operation (the matrix to be transposed) is transposed to obtain the correct calculation result, which helps to improve the accuracy of the chip. It can also be understood that the scale of the matrix to be transposed can be block_n×m, and the scale of the result matrix can be m×block_n.

[0047] In the embodiment of the present disclosure, an output matrix can be obtained according to one or more result matrices. Figure 3 As shown, the result matrix C301 can be obtained by performing matrix operations based on the first matrix A30 and multiple sub-matrices to be processed of the matrix B301 to be processed. The result matrix C302 can be obtained by performing matrix operations based on the first matrix A30 and multiple sub-matrices to be processed of the matrix B302 to be processed. According to the result matrix C300, the result matrix C301 and the result matrix C302, the output matrix C30 can be obtained.

[0048] It can be understood that the above description uses the example that the amount of data in the first matrix is ​​greater than the capacity of the first storage space and less than the capacity of the second storage space to illustrate the present disclosure, but the present disclosure is not limited thereto, which will be described below.

[0049] In other embodiments, in other implementations of the above operation S210, in response to determining that the amount of data of the first matrix is ​​greater than the capacity of the first storage space, the amount of data of the first matrix is ​​less than or equal to the capacity of the second storage space, and the amount of data of the second matrix is ​​greater than the capacity of the second storage space, the first matrix is ​​moved to the second storage space. For example, if the amount of data of the above-mentioned first matrix A30 is greater than the capacity of the first storage space bufferA30, the amount of data of the first matrix A30 is less than or equal to the capacity of the second storage space bufferB30, and the amount of data of the second matrix B30 is greater than the capacity of the second storage space bufferB30, the first matrix A30 can be moved to the second storage space bufferB30. Through the embodiments of the present disclosure, when the amount of data of the second matrix is ​​greater than the capacity of the second storage space, the first matrix is ​​stored in the second storage space. It can be applicable to the scenario where the amount of data of the second matrix is ​​small.

[0050] It can be understood that the method of the present disclosure is described above, and some ways of determining the sub-matrix to be processed will be described below.

[0051] Figure 4A is a schematic flowchart of determining an initial sub-matrix according to an embodiment of the present disclosure.

[0052] like Figure 4A As shown, method 401 may be performed before the above method 200, and method 401 may include operations S4001 to S4008, which will be described below.

[0053] In operation S4001, a matrix to be processed of the second matrix is ​​determined according to the data amount of the first matrix and the data amount of the second matrix.

[0054] For example, the second matrix is ​​split into a plurality of matrices to be processed according to the number of processor cores. The number of matrices to be processed may be greater than or equal to the number of processor cores. The size of the second matrix may be n×k. The size of the matrix to be processed may be block_n×k. block_n may be less than or equal to n.

[0055] In operation S4002, the first matrix and the matrix to be processed are split multiple times to obtain multiple first initial sub-matrix groups and multiple initial sub-matrix groups to be processed.

[0056] In the disclosed embodiment, according to a plurality of different first scale information and a plurality of different second scale information, the first matrix and the matrix to be processed are split multiple times, respectively, to obtain a plurality of first initial submatrix groups corresponding to the plurality of first scale information and a plurality of initial submatrix groups to be processed corresponding to the plurality of second scale information. For example, according to a plurality of different first scale information, the first matrix can be split multiple times. According to a plurality of different second scale information, the matrix to be processed can be split multiple times. The first initial submatrix group includes a plurality of first initial submatrices. The initial submatrix group to be processed includes a plurality of initial submatrices to be processed. The scale of the initial submatrix to be processed can be, for example, tile_n'×k, and the scale of the first initial submatrix can be, for example, k×tile_m'.

[0057] In an embodiment of the present disclosure, the first scale information may include the first number of rows and the first number of columns. The first number of columns may be different between different first scale information. The second scale information may include the second number of columns and the second number of rows. The second number of rows may be different between different second scale information. For example, a first initial value may be set as a first number of columns. Next, one or more times the preset first step length may be added to the first initial value to obtain multiple first column numbers. The first column number with the largest value is less than or equal to m. A second initial value may also be set as a second row number. Next, one or more times the preset second step length may be added to the second initial value to obtain multiple second row numbers. The second row number with the largest value is less than or equal to block_n.

[0058] Next, multiple scale information combinations consisting of multiple first scale information and multiple second scale information can be verified to determine whether the scale information combination meets the preset conditions. It can be understood that the capacity of the first cache space, the second cache space, the first storage space, and the second storage space can be determined. Therefore, in order to determine whether the above-mentioned initial submatrix to be processed, the initial submatrix to be transposed, and the initial result submatrix can be stored in the corresponding storage space, it can be determined whether the scale information combination meets the preset conditions.

[0059] In some embodiments, there may be multiple preset conditions. The multiple preset conditions may include at least one of the following: the capacity of the second cache space is a multiple of the data volume of the submatrix to be processed indicated by the second scale information. The capacity of the first storage space is a multiple of the data volume of the submatrix to be processed indicated by the second scale information. The multiple may be multiple integer multiples. For example, the multiple may be at least 2 times.

[0060] In some embodiments, the scale of the result matrix is ​​related to the scale of the first matrix and the matrix to be processed. Based on the scale of the first initial submatrix and the scale of the initial submatrix to be processed, the scale of the initial submatrix to be transposed and the scale of the initial result submatrix can be determined. The capacity of the third cache space and the capacity of the fourth cache space are also limited. Thus, the above-mentioned multiple preset conditions may also include: the capacity of the third cache space is multiple times the amount of data of the initial submatrix to be transposed corresponding to the first scale information and the second scale information. The capacity of the fourth cache space is multiple times the amount of data of the initial result submatrix corresponding to the first scale information and the second scale information. It will be described below in conjunction with operations S4003 to S4006.

[0061] In operation S4003, it is determined whether the capacity capacity_bufferA of the first storage space is at least twice the data amount size_b of the submatrix to be processed indicated by the second scale information. In response to determining that the capacity of the first storage space is at least twice the data amount of the submatrix to be processed indicated by the second scale information, operation S4007 is performed. In response to determining that the capacity of the first storage space is not at least twice the data amount of the submatrix to be processed indicated by the second scale information, the scale information combination corresponding to the second scale information is deleted, and operations S4003 to S4006 are performed on the scale information combination other than the scale information combination.

[0062] In operation S4004, it is determined whether the capacity capacity_cache2 of the second cache space is at least twice the data amount size_b of the submatrix to be processed indicated by the second scale information. In response to determining that the capacity of the second cache space is at least twice the data amount of the submatrix to be processed indicated by the second scale information, operation S4007 is performed. In response to determining that the capacity of the second cache space is not at least twice the data amount of the submatrix to be processed indicated by the second scale information, the scale information combination corresponding to the second scale information is deleted, and operations S4003 to S4006 are performed on the scale information combination other than the scale information combination.

[0063] In operation S4005, it is determined whether the capacity capacity_cache3 of the third cache space is at least twice the data volume of the initial submatrix to be transposed size_cT corresponding to the first scale information and the second scale information. In response to determining that the capacity of the third cache space is at least twice the data volume of the initial submatrix to be transposed, operation S4007 is performed. In response to determining that the capacity of the third cache space is not at least twice the data volume of the initial submatrix to be transposed, the scale information combination corresponding to the second scale information is deleted, and operations S4003 to S4006 are performed on the scale information combination other than the scale information combination.

[0064] In operation S4006, it is determined whether the capacity capacity_cache4 of the fourth cache space is at least twice the data amount size_c of the initial result submatrix corresponding to the first scale information and the second scale information. In response to determining that the capacity of the fourth cache space is at least twice the data amount of the initial result submatrix, operation S4007 is performed. In response to determining that the capacity of the fourth cache space is not at least twice the data amount of the initial result submatrix, the scale information combination corresponding to the second scale information is deleted, and operations S4003 to S4006 are performed on the scale information combination other than the scale information combination.

[0065] In operation S4007, the scale information combination that meets the preset condition is saved.

[0066] In operation S4008, it is determined whether all scale information combinations have been verified. In response to determining that all scale information combinations have been verified, at least one scale information combination that meets the preset condition is output. In response to determining that all scale information combinations have not been verified, it is returned to operation S4003 to operation S4007 to verify the unverified scale information combination.

[0067] It can be understood that a plurality of scale information combinations are determined above, and some methods of determining sub-matrices will be described below.

[0068] In the embodiment of the present disclosure, the above method may further include: performing multiple matrix operations based on multiple first initial sub-matrix groups and multiple initial sub-matrix groups to be processed to obtain multiple resource overhead information, where the resource overhead information corresponds to the first scale information and the second scale information. Determine the target resource overhead information based on the multiple resource overhead information. For example, performing multiple matrix operations based on multiple first initial sub-matrix groups and multiple initial sub-matrix groups to be processed to obtain multiple initial sub-matrix groups to be transposed and multiple initial result sub-matrix groups, where the initial sub-matrix groups to be transposed include multiple initial sub-matrices to be transposed, and the initial result sub-matrix groups include multiple initial result sub-matrices. Figure 4B Provide explanation.

[0069] Figure 4B is a schematic flowchart of determining a sub-matrix according to an embodiment of the present disclosure.

[0070] like Figure 4B As shown, method 402 may be performed after method 401 and before method 200. Method 402 may include operations S4009 to S4012.

[0071] In operation S4009, the current combination of scale information to be processed is obtained. For example, the current combination of scale information to be processed can be determined from the above multiple combinations of scale information that meet the preset conditions.

[0072] In operation S4010, matrix operations are performed according to the current combination of scale information to be processed. For example, one or more matrix operations are performed according to the first initial sub-matrix group and the initial sub-matrix group to be processed corresponding to the current combination of scale information to be processed, and resource overhead information and one or more initial sub-matrices to be transposed corresponding to the current combination of scale information to be processed are obtained. One or more initial sub-matrices to be transposed are transposed to obtain one or more initial result sub-matrices. Resource overhead may include computing resource overhead, bandwidth overhead, memory access overhead, etc.

[0073] In operation S4011, the current optimal resource overhead information is determined. For example, the resource overhead information with the lowest resource overhead can be used as the current optimal resource overhead information. It can be understood that if any scale information combination has not been processed before the current optimal resource overhead information, the current resource overhead information can be used as the optimal one. If one or more scale information combinations have been processed before the current optimal resource overhead information, and the previous optimal resource overhead information has been saved, the information with the smaller resource overhead can be used as the current optimal resource overhead information from the previous optimal resource overhead information and the current resource overhead information.

[0074] In operation S4012, it is determined whether all scale information combinations that meet the preset conditions have been processed. In response to all scale information combinations that meet the preset conditions being processed, the target resource overhead information is output. In response to not all scale information combinations that meet the preset conditions being processed, the process returns to operation S4009.

[0075] In an embodiment of the present disclosure, the above method may further include: determining a plurality of first sub-matrices of the first matrix and a plurality of to-be-processed sub-matrices of the second matrix according to first scale information and second scale information corresponding to the target resource overhead information.

[0076] Through the embodiments of the present disclosure, multiple sub-matrices to be processed can be stored in the first storage space, multiple sub-matrices to be processed can also be stored in the second cache space, multiple sub-matrices to be transposed can be stored in the third cache space, and multiple result sub-matrices can be stored in the fourth cache space, which can better realize pipeline parallelism, fully utilize the computing power within the computing unit, and improve computing efficiency.

[0077] Through the embodiments of the present disclosure, target resource overhead information is determined, which can reduce computing costs while improving computing efficiency.

[0078] It can be understood that some methods of splitting a matrix are described above, and the parallel computing method of the computing unit of the present disclosure will be described below.

[0079] Figure 5is a schematic diagram of a data processing method according to an embodiment of the present disclosure.

[0080] In the embodiments of the present disclosure, in some implementations of the above operation S210, in response to determining that the data amount of the first matrix is ​​greater than the capacity of the first storage space and less than or equal to the capacity of the second storage space, moving the first matrix to the second storage space includes: loading the first matrix from the global storage unit to the first cache space. In response to determining that the data amount of the first matrix is ​​greater than the capacity of the first storage space and less than or equal to the capacity of the second storage space, loading a target matrix to be processed from the multiple matrices to be processed from the global storage unit to the second cache space, and synchronously moving the first matrix from the first cache space to the second storage space. Figure 5 As shown, the first matrix A50 can be loaded from the global storage unit to the first cache space. Next, the matrix B500 to be processed can be loaded from the global storage unit to the second cache space, and the first matrix A50 can be synchronously moved to the second storage space. It can be understood that the above description of the first matrix A30 and the second matrix B30 is also applicable to the first matrix A50 and the second matrix B50. Through the embodiment of the present disclosure, the second matrix can be moved from the global storage unit synchronously with the first matrix being moved to the first storage space, which can effectively avoid idle memory resources, improve memory resource utilization, and also help improve the overall task execution efficiency of the chip.

[0081] In the embodiment of the present disclosure, the second cache space includes a plurality of cache subspaces. For example, the cache subspace can store a matrix to be processed. There can be multiple submatrices to be processed of the matrix to be processed. For example, there can be 2 cache subspaces.

[0082] In the embodiment of the present disclosure, the first storage space includes a plurality of storage subspaces. The storage subspaces can store submatrices to be processed.

[0083] In the embodiment of the present disclosure, in some implementations of the above operation S220, moving the to-be-processed submatrix of the to-be-processed matrix to the first storage space includes: moving the to-be-processed submatrix of the target to-be-processed matrix to the first storage space, and synchronously moving the subsequent to-be-processed matrix of the target to-be-processed matrix to an available cache subspace among the multiple cache subspaces of the second cache space. Moving the target to-be-processed submatrix among the multiple to-be-processed submatrices to the storage subspace of the first storage space. Figure 5As shown, after loading the matrix B500 to be processed into the first cache subspace of the second cache space, the submatrix B5001 to be processed of the matrix B500 to be processed can be moved to the first storage subspace of the first storage space, and the matrix B501 to be processed can be synchronously loaded from the global storage unit to the second cache subspace of the second cache space. It can be understood that the matrix B500 to be processed can be used as the target matrix to be processed, and the matrix B501 to be processed can be used as the matrix to be processed later. In the case where the first cache subspace of the second cache space stores the target submatrix to be processed, the second cache subspace of the second cache space can be used as an available cache subspace. Through the embodiment of the present disclosure, the moving of the submatrix to be processed and the moving of the matrix to be processed later can be performed synchronously, further effectively utilizing the bandwidth resources and memory access resources of the chip, and after performing related operations using the target matrix to be processed, the matrix to be processed later can be quickly used to perform corresponding operations, which can effectively improve the computing efficiency of the chip.

[0084] In the disclosed embodiment, moving the to-be-processed submatrix of the to-be-processed matrix to the first storage space also includes: moving the subsequent to-be-processed submatrix of the target to-be-processed submatrix to an available storage subspace among the plurality of storage subspaces. For example, the to-be-processed submatrix B5002 of the to-be-processed matrix B5000 can be moved to an available second storage subspace. It can be understood that the to-be-processed submatrix B5001 can be used as the target to-be-processed submatrix, and the to-be-processed submatrix B5002 can be used as the subsequent to-be-processed submatrix.

[0085] In the embodiment of the present disclosure, the first matrix may correspond to a plurality of first sub-matrices. The scale of the first sub-matrices may be k×tile_n.

[0086] In some implementations of the above operation S230, performing matrix operations according to the first matrix and the submatrix to be processed of the matrix to be processed to obtain the result matrix includes: moving the submatrix to be processed later to an available storage subspace in the plurality of storage subspaces of the first storage space, and synchronously performing matrix operations according to the first target submatrix of the first matrix and the target submatrix to be processed by using a computing unit to obtain a target submatrix to be transposed in the plurality of submatrices to be transposed of the matrix to be transposed. Figure 5As shown, while the submatrix to be processed B5002 is moved to the available storage subspace of the first storage space, the first target submatrix and the submatrix to be processed B5001 can be loaded using the systolic array to perform calculations to obtain the submatrix to be transposed CT5001. The scale of the submatrix to be transposed CT5001 can be tile_n×tile_m. The submatrix to be transposed CT5001 can be stored in the first cache subspace of the third cache space. It can be understood that the submatrix to be transposed CT5001 can be used as the target submatrix to be transposed. Through the embodiments of the present disclosure, the movement of the submatrix to be processed and the matrix operation based on the target submatrix to be processed can be performed synchronously, which can improve the utilization of computing resources and memory access resources and improve the computing efficiency of the chip.

[0087] In some implementations of the above operation S230, according to the first matrix and the to-be-processed submatrix of the to-be-processed matrix, performing matrix operations to obtain a result matrix includes: transposing the target to-be-transposed submatrix to obtain a target result submatrix among the multiple result submatrices of the result matrix. The target result submatrix is ​​stored in one of the multiple cache subspaces of the fourth cache space. Figure 5 As shown, the submatrix to be transposed CT5001 is transposed to obtain a result submatrix C5001. The result submatrix C5001 can be stored in the first cache subspace of the fourth cache space. It can be understood that the result submatrix C5001 can be used as the target result submatrix.

[0088] In the embodiment of the present disclosure, performing matrix operations according to the first matrix and the submatrix to be processed of the matrix to be processed to obtain the result matrix further includes: transposing the target submatrix to be transposed, and synchronously performing matrix operations according to the first target submatrix and the subsequent submatrix to be processed using a computing unit to obtain the subsequent submatrix to be transposed of the target submatrix to be transposed. Figure 5 As shown, while the submatrix to be transposed CT5001 is transposed, the submatrix to be processed B5002 can be loaded by the systolic array to perform calculations to obtain the submatrix to be transposed CT5002. It can be understood that the submatrix to be transposed CT5002 can be used as the submatrix to be transposed later. Through the embodiments of the present disclosure, the systolic unit and the transposition unit can process data synchronously, which can effectively improve the degree of parallelism between different computing units and fully improve the computing efficiency of the chip.

[0089] In the embodiment of the present disclosure, performing matrix operations according to the first matrix and the to-be-processed submatrix of the to-be-processed matrix to obtain a result matrix further includes: storing the target result submatrix in one of the plurality of cache subspaces of the fourth cache space, and synchronously transposing the subsequent to-be-transposed submatrix to obtain a subsequent result submatrix of the target result submatrix. Figure 5As shown, while storing the result sub-matrix C5001 in the first cache sub-space of the fourth cache space, the sub-matrix to be transposed CT5002 is transposed to obtain the result sub-matrix C5002. The result sub-matrix C5002 can be stored in the second cache sub-space of the fourth cache space. It can be understood that the result sub-matrix C5002 can be used as the subsequent result sub-matrix. The result sub-matrix can also be stored in the global storage unit. Through the embodiment of the present disclosure, transposition and storage can be performed in parallel, further improving the utilization rate of computing resources and memory access resources.

[0090] In the embodiment of the present disclosure, moving the to-be-processed submatrix of the to-be-processed matrix to the first storage space further includes: in response to having performed matrix operations according to multiple to-be-processed submatrices of the target to-be-processed submatrix, moving the to-be-processed submatrix of the subsequent to-be-processed matrix from the second cache space to an available storage subspace among the multiple storage subspaces. Figure 5 As shown, taking the matrix to be processed corresponding to two sub-matrices to be processed as an example, when the systolic array is used to calculate according to the first target sub-matrix and the sub-matrix to be processed B5001, the first storage subspace can be used as an available storage subspace. The sub-matrix to be processed B5011 of the matrix to be processed B501 can be moved to the first storage subspace of the first storage space. It can be understood that the processing method of the matrix to be processed B501 is the same or similar to the processing method of the matrix to be processed B500, and the present disclosure will not repeat it here.

[0091] Through the embodiments of the present disclosure, the storage space and the cache space are divided into subspaces, so that matrix operations can be parallelized at the pipeline level, and the computing power of the computing unit can be fully utilized to improve computing efficiency.

[0092] It can be understood that the method of the present disclosure is described above in conjunction with the first target sub-matrix, and the first subsequent sub-matrix of the first target sub-matrix will be described below.

[0093] In an embodiment of the present disclosure, performing matrix operations according to the first matrix and the submatrix to be processed of the matrix to be processed to obtain a result matrix further includes: in response to determining that the matrix operation has been performed according to the first target submatrix in the multiple first submatrices and the multiple submatrices to be processed of the matrix to be processed, performing matrix operations according to the first subsequent submatrix of the first target submatrix and the submatrix to be processed of the matrix to be processed to obtain a result matrix. For example, if the matrix operation has been performed according to the first target submatrix and the multiple submatrices to be processed of the matrix to be processed B500, the first subsequent submatrix can be loaded from the second storage space to perform the matrix operation. It can be understood that the method of calculating using the first subsequent submatrix is ​​the same or similar to the method of calculating using the first target submatrix, and the present disclosure will not repeat them here.

[0094] In the disclosed embodiment, performing matrix operations according to the first matrix and the submatrix to be processed of the matrix to be processed to obtain the result matrix further includes: in response to having performed matrix operations according to the multiple submatrices to be processed in the first storage space, performing matrix operations according to the first subsequent submatrix and the multiple submatrices to be processed in the first storage space. For example, the time overhead of loading the matrix from the first storage space or the second storage space may be less than the overhead of repeatedly moving the submatrix to be processed. Thus, after performing matrix operations using the first target submatrix, matrix operations may be performed using the first subsequent submatrix and the multiple submatrices to be processed in the first storage space.

[0095] It can be understood that the method of the present disclosure is described above, and the device of the present disclosure will be described below.

[0096] Figure 6 is a block diagram of a data processing device according to an embodiment of the present disclosure.

[0097] like Figure 6 As shown, the device 600 may include a first buffer unit 610 , a second buffer unit 620 , a data handling unit 630 and a calculation module 640 .

[0098] The first buffer unit 610 includes a first storage space.

[0099] The second buffer unit 620 includes a second storage space.

[0100] The data transport unit 630 is configured to: in response to determining that the data amount of the first matrix is ​​greater than the capacity of the first storage space and less than or equal to the capacity of the second storage space, transport the first matrix to the second storage space; and transport the to-be-processed sub-matrix of the to-be-processed matrix to the first storage space. The to-be-processed matrix is ​​determined according to the second matrix, and the data amount of the second matrix is ​​greater than or equal to the to-be-processed matrix.

[0101] The calculation module 640 is configured to perform matrix operations according to the first matrix and the sub-matrix to be processed of the matrix to be processed to obtain a result matrix.

[0102] In some embodiments, the data transport unit is further configured to perform the following operations to transport the first matrix to the second storage space in response to determining that the amount of data of the first matrix is ​​greater than the capacity of the first storage space and less than or equal to the capacity of the second storage space: in response to determining that the amount of data of the first matrix is ​​greater than the capacity of the first storage space, the amount of data of the first matrix is ​​less than or equal to the capacity of the second storage space, and the amount of data of the second matrix is ​​greater than the capacity of the second storage space, transport the first matrix to the second storage space.

[0103] In some embodiments, the calculation module includes a calculation unit and a transposition unit. The calculation unit is further configured to: perform matrix operations using the calculation unit according to the first matrix and the sub-matrix to be processed to obtain the matrix to be transposed. The transposition unit is further configured to: transpose the matrix to be transposed to obtain a result matrix.

[0104] In some embodiments, the second matrix corresponds to a plurality of matrices to be processed. The data handling unit is further configured to perform the following operations in response to determining that the amount of data of the first matrix is ​​greater than the capacity of the first storage space and less than or equal to the capacity of the second storage space, to carry the first matrix to the second storage space: in response to determining that the amount of data of the first matrix is ​​greater than the capacity of the first storage space and less than or equal to the capacity of the second storage space, to load a target matrix to be processed from the plurality of matrices to be processed from the global storage unit to the second cache space, and to synchronously carry the first matrix from the first cache space to the second storage space.

[0105] In some embodiments, the second cache space includes multiple cache subspaces, and the data transfer unit is further configured to perform the following operations to transfer the processed submatrix of the matrix to be processed to the first storage space: transfer the processed submatrix of the target matrix to be processed to the first storage space, and synchronously transfer the subsequent processed matrix of the target matrix to be processed to an available cache subspace among the multiple cache subspaces.

[0106] In some embodiments, there are multiple sub-matrices to be processed of the target matrix to be processed, the first storage space includes multiple storage sub-spaces, and the data handling unit is further configured to perform the following operations to carry the sub-matrix to be processed of the matrix to be processed to the first storage space: carry the target sub-matrix to be processed among the multiple sub-matrices to be processed to the storage sub-space. Carry the subsequent sub-matrix to be processed of the target sub-matrix to be processed to an available storage sub-space among the multiple storage sub-spaces.

[0107] In some embodiments, the calculation module includes a calculation unit and a transposition unit, and the calculation unit is further configured to: synchronously with the data handling unit carrying the subsequent to-be-processed submatrix, according to the first target submatrix and the target to-be-processed submatrix of the first matrix, use the calculation unit to perform matrix operations to obtain the target to-be-transposed submatrix among the multiple to-be-transposed submatrices of the to-be-transposed matrix. The calculation unit is also configured to: store the target to-be-transposed submatrix in one of the multiple cache subspaces of the third cache space. The transposition unit is also configured to: transpose the target to-be-transposed submatrix to obtain the target result submatrix among the multiple result submatrices of the result matrix; and store the target result submatrix in one of the multiple cache subspaces of the fourth cache space.

[0108] In some embodiments, the computing unit is further configured to: synchronously with the transposition unit transposing the target to-be-transposed submatrix, according to the first target submatrix and the subsequent to-be-processed submatrix, use the computing unit to perform matrix operations to obtain the subsequent to-be-transposed submatrix of the target to-be-transposed submatrix. The transposition unit is further configured to: store the target result submatrix in one of the plurality of cache subspaces of the fourth cache space, synchronously transpose the subsequent to-be-transposed submatrix, and obtain the subsequent result submatrix of the target result submatrix.

[0109] In some embodiments, the data handling unit is further configured to perform the following operations to handle the sub-matrix to be processed of the matrix to be processed to the first storage space: in response to matrix operations having been performed based on multiple sub-matrices to be processed of the target sub-matrix to be processed, handle the sub-matrix to be processed of the subsequent matrix to be processed from the second cache space to an available storage sub-space among the multiple storage sub-spaces.

[0110] In some embodiments, the first matrix includes multiple first sub-matrices, and the computing module is further configured to: in response to determining that matrix operations have been performed based on the first target sub-matrix and multiple sub-matrices to be processed of the matrix to be processed among the multiple first sub-matrices, perform matrix operations based on the first subsequent sub-matrix of the first target sub-matrix and the sub-matrix to be processed of the matrix to be processed to obtain a result matrix.

[0111] In some embodiments, the first matrix includes a plurality of first sub-matrices, the matrix to be processed has a plurality of sub-matrices to be processed, the result matrix corresponds to the plurality of result sub-matrices, and the plurality of first sub-matrices of the first matrix and the plurality of sub-matrices to be processed of the second matrix are determined by the following operations: according to the data amount of the first matrix and the data amount of the second matrix, the matrix to be processed of the second matrix is ​​determined. According to a plurality of different first scale information and a plurality of different second scale information, the first matrix and the matrix to be processed are split multiple times respectively to obtain a plurality of first initial sub-matrix groups corresponding to the plurality of first scale information and a plurality of initial sub-matrix groups to be processed corresponding to the plurality of second scale information. The first initial sub-matrix group includes a plurality of first initial sub-matrices, and the initial sub-matrix group to be processed includes a plurality of initial sub-matrices to be processed. According to the plurality of first initial sub-matrix groups and the plurality of initial sub-matrix groups to be processed, multiple matrix operations are performed to obtain a plurality of resource overhead information, and the resource overhead information corresponds to the first scale information and the second scale information. According to the plurality of resource overhead information, the target resource overhead information is determined. According to the first scale information and the second scale information corresponding to the target resource overhead information, the plurality of first sub-matrices of the first matrix and the plurality of sub-matrices to be processed of the second matrix are determined.

[0112] In some embodiments, the matrix to be transposed corresponds to a plurality of sub-matrices to be transposed, the sub-matrices to be transposed are stored in the third cache space, and the result sub-matrix is ​​stored in the fourth cache space. According to the plurality of first initial sub-matrix groups and the plurality of initial sub-matrix groups to be processed, performing multiple matrix operations includes: according to the plurality of first initial sub-matrix groups and the plurality of initial sub-matrix groups to be processed, performing multiple matrix operations to obtain a plurality of initial sub-matrix groups to be transposed and a plurality of initial result sub-matrix groups, the initial sub-matrix groups to be transposed include a plurality of initial sub-matrixes to be transposed, and the initial result sub-matrix groups include a plurality of initial result sub-matrices. The capacity of the second cache space is a multiple of the amount of data of the sub-matrix to be processed indicated by the second scale information, the capacity of the first storage space is a multiple of the amount of data of the sub-matrix to be processed indicated by the second scale information, the capacity of the third cache space is a multiple of the amount of data of the initial sub-matrix to be transposed corresponding to the first scale information and the second scale information, and the capacity of the fourth cache space is a multiple of the amount of data of the initial result sub-matrix corresponding to the first scale information and the second scale information.

[0113] Figure 7 is a block diagram of an electronic device according to an embodiment of the present disclosure.

[0114] like Figure 7 As shown, the device 70 may include a data processing device 700. The data processing device 700 may be the device 600 described above.

[0115] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0116] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0117] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0118] like Figure 8As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 to a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0119] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0120] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSP), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as data processing methods. For example, in some embodiments, the data processing method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the data processing method in any other appropriate manner (eg, by means of firmware).

[0121] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system on chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0122] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0123] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories, read-only memories, erasable programmable read-only memories (EPROM) or flash memories, optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0124] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) display or a liquid crystal display (LCD)) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0125] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a Local Area Network (LAN), a Wide Area Network (WAN), and the Internet.

[0126] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0127] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0128] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A data processing method, comprising: In response to determining that the amount of data of the first matrix is ​​greater than the capacity of the first storage space and less than or equal to the capacity of the second storage space, moving the first matrix to the second storage space; Moving a submatrix to be processed of a matrix to be processed to the first storage space, wherein the matrix to be processed is determined according to a second matrix, and the amount of data of the second matrix is ​​greater than or equal to the matrix to be processed; Matrix operations are performed according to the first matrix and the sub-matrix to be processed of the matrix to be processed to obtain a result matrix.

2. The method according to claim 1, wherein: In response to determining that the amount of data of the first matrix is ​​greater than the capacity of the first storage space and less than or equal to the capacity of the second storage space, moving the first matrix to the second storage space includes: In response to determining that the data amount of the first matrix is ​​greater than the capacity of the first storage space, the data amount of the first matrix is ​​less than or equal to the capacity of the second storage space, and the data amount of the second matrix is ​​greater than the capacity of the second storage space, the first matrix is ​​moved to the second storage space.

3. The method according to claim 1, wherein: The performing matrix operation according to the first matrix and the to-be-processed sub-matrix to be processed to obtain a result matrix comprises: According to the first matrix and the sub-matrix to be processed, the calculation unit is used to perform matrix operations to obtain a matrix to be transposed; The matrix to be transposed is transposed to obtain the result matrix.

4. The method according to claim 1, wherein: The second matrix corresponds to a plurality of matrices to be processed, In response to determining that the amount of data of the first matrix is ​​greater than the capacity of the first storage space and less than or equal to the capacity of the second storage space, moving the first matrix to the second storage space includes: Loading the first matrix from a global storage unit into a first cache space; In response to determining that the data amount of the first matrix is ​​greater than the capacity of the first storage space and less than or equal to the capacity of the second storage space, a target matrix to be processed among the multiple matrices to be processed is loaded from the global storage unit to the second cache space, and the first matrix is ​​synchronously moved from the first cache space to the second storage space.

5. The method according to claim 4, wherein: The second cache space includes a plurality of cache subspaces, The moving the to-be-processed sub-matrix of the to-be-processed matrix to the first storage space comprises: The to-be-processed submatrix of the target to-be-processed matrix is ​​moved to the first storage space, and the subsequent to-be-processed matrix of the target to-be-processed matrix is ​​synchronously moved to an available cache subspace among the plurality of cache subspaces.

6. The method according to claim 4, wherein: The target matrix to be processed has multiple sub-matrices to be processed, and the first storage space includes multiple storage sub-spaces. The moving the to-be-processed sub-matrix of the to-be-processed matrix to the first storage space comprises: Moving a target submatrix to be processed among the plurality of submatrices to be processed to the storage subspace; The subsequent sub-matrix to be processed of the target sub-matrix to be processed is moved to an available storage sub-space among the plurality of storage sub-spaces.

7. The method according to claim 6, wherein: The performing matrix operation according to the first matrix and the to-be-processed sub-matrix of the to-be-processed matrix to obtain a result matrix comprises: The submatrix to be processed later is moved to an available storage subspace among the plurality of storage subspaces, and a matrix operation is performed by a computing unit synchronously according to the first target submatrix of the first matrix and the target submatrix to be processed, so as to obtain a target submatrix to be transposed among the plurality of submatrices to be transposed of the matrix to be transposed; storing the target to-be-transposed submatrix in one of a plurality of cache subspaces of a third cache space; Transposing the target to-be-transposed submatrix to obtain a target result submatrix among a plurality of result submatrices of the result matrix; The target result sub-matrix is ​​stored in one of a plurality of cache sub-spaces of a fourth cache space.

8. The method according to claim 7, wherein: The performing matrix operation according to the first matrix and the to-be-processed sub-matrix of the to-be-processed matrix to obtain a result matrix further comprises: Transposing the target submatrix to be transposed, and synchronously performing matrix operations using the computing unit according to the first target submatrix and the subsequent submatrix to be processed, to obtain a subsequent submatrix to be transposed of the target submatrix to be transposed; The target result submatrix is ​​stored in one of a plurality of cache subspaces of a fourth cache space, and the subsequent submatrix to be transposed is synchronously transposed to obtain a subsequent result submatrix of the target result submatrix.

9. The method according to claim 6, wherein: The moving the to-be-processed sub-matrix of the to-be-processed matrix to the first storage space comprises: In response to having performed matrix operations according to the plurality of to-be-processed sub-matrices of the target to-be-processed sub-matrix, the to-be-processed sub-matrix of the subsequent to-be-processed matrix is ​​moved from the second cache space to an available storage sub-space among the plurality of storage sub-spaces.

10. The method according to claim 1, wherein: The first matrix includes a plurality of first sub-matrices, The performing matrix operation according to the first matrix and the to-be-processed sub-matrix of the to-be-processed matrix to obtain a result matrix further comprises: In response to determining that a matrix operation has been performed based on a first target submatrix among the plurality of first submatrices and a plurality of submatrices to be processed of the matrix to be processed, a matrix operation is performed based on a first subsequent submatrix of the first target submatrix and a submatrix to be processed of the matrix to be processed to obtain the result matrix.

11. The method according to claim 3, wherein: The first matrix includes a plurality of first sub-matrices, the matrix to be processed includes a plurality of sub-matrices to be processed, and the result matrix corresponds to a plurality of result sub-matrices. The method further comprises: Determining a matrix to be processed of the second matrix according to the data amount of the first matrix and the data amount of the second matrix; According to a plurality of different first scale information and a plurality of different second scale information, the first matrix and the matrix to be processed are split multiple times respectively to obtain a plurality of first initial sub-matrix groups corresponding to the plurality of first scale information and a plurality of initial sub-matrix groups to be processed corresponding to the plurality of second scale information, wherein the first initial sub-matrix group includes a plurality of first initial sub-matrices, and the initial sub-matrix group to be processed includes a plurality of initial sub-matrices to be processed; Perform multiple matrix operations according to the multiple first initial sub-matrix groups and the multiple initial to-be-processed sub-matrix groups to obtain multiple resource overhead information, where the resource overhead information corresponds to the first scale information and the second scale information; Determining target resource overhead information according to the plurality of resource overhead information; According to the first scale information and the second scale information corresponding to the target resource overhead information, a plurality of first sub-matrices of the first matrix and a plurality of to-be-processed sub-matrices of the second matrix are determined.

12. The method according to claim 11, wherein: The matrix to be transposed corresponds to a plurality of sub-matrices to be transposed, the sub-matrices to be transposed are stored in a third cache space, and the result sub-matrix is ​​stored in a fourth cache space. The performing a plurality of matrix operations according to the plurality of the first initial sub-matrix groups and the plurality of the initial sub-matrix groups to be processed comprises: According to the plurality of the first initial sub-matrix groups and the plurality of the initial sub-matrix groups to be processed, a plurality of matrix operations are performed to obtain a plurality of initial sub-matrix groups to be transposed and a plurality of initial result sub-matrix groups, wherein the initial sub-matrix groups to be transposed include a plurality of initial sub-matrices to be transposed, and the initial result sub-matrix groups include a plurality of initial result sub-matrices, The capacity of the second cache space is a multiple of the data amount of the submatrix to be processed indicated by the second scale information, the capacity of the first storage space is a multiple of the data amount of the submatrix to be processed indicated by the second scale information, the capacity of the third cache space is a multiple of the data amount of the initial submatrix to be transposed corresponding to the first scale information and the second scale information, and the capacity of the fourth cache space is a multiple of the data amount of the initial result submatrix corresponding to the first scale information and the second scale information.

13. A data processing device, comprising: The first buffer unit, The second buffer unit, A data transport unit is configured to: in response to determining that the amount of data of the first matrix is ​​greater than the capacity of the first storage space and less than or equal to the capacity of the second storage space, transport the first matrix to the second storage space; and moving a submatrix to be processed of a matrix to be processed to the first storage space, wherein the matrix to be processed is determined according to a second matrix, the amount of data of the second matrix is ​​greater than or equal to the matrix to be processed, the first storage space is located in the first buffer unit, and the second storage space is located in the second buffer unit; The calculation module is configured to: perform matrix operations according to the first matrix and the sub-matrix to be processed of the matrix to be processed to obtain a result matrix.

14. The device according to claim 13, wherein: The data transport unit is further configured to perform the following operations to transport the first matrix to the second storage space in response to determining that the amount of data of the first matrix is ​​greater than the capacity of the first storage space and less than or equal to the capacity of the second storage space: In response to determining that the data amount of the first matrix is ​​greater than the capacity of the first storage space, the data amount of the first matrix is ​​less than or equal to the capacity of the second storage space, and the data amount of the second matrix is ​​greater than the capacity of the second storage space, the first matrix is ​​moved to the second storage space.

15. The device according to claim 13, wherein: The calculation module includes a calculation unit and a transposition unit. The calculation unit is further configured to: perform matrix operations using the calculation unit according to the first matrix and the sub-matrix to be processed to obtain a matrix to be transposed; The transposition unit is further configured to: transpose the matrix to be transposed to obtain the result matrix.

16. The device according to claim 13, wherein: The second matrix corresponds to a plurality of matrices to be processed, The data transport unit is further configured to perform the following operations to transport the first matrix to the second storage space in response to determining that the amount of data of the first matrix is ​​greater than the capacity of the first storage space and less than or equal to the capacity of the second storage space: In response to determining that the data amount of the first matrix is ​​greater than the capacity of the first storage space and less than or equal to the capacity of the second storage space, a target matrix to be processed among the multiple matrices to be processed is loaded from the global storage unit to the second cache space, and the first matrix is ​​synchronously moved from the first cache space to the second storage space.

17. The device according to claim 16, wherein: The second cache space includes a plurality of cache subspaces, The data transport unit is further configured to perform the following operations to transport the to-be-processed sub-matrix of the to-be-processed matrix to the first storage space: The to-be-processed submatrix of the target to-be-processed matrix is ​​moved to the first storage space, and the subsequent to-be-processed matrix of the target to-be-processed matrix is ​​synchronously moved to an available cache subspace among the plurality of cache subspaces.

18. The device according to claim 16, wherein: The target matrix to be processed has multiple sub-matrices to be processed, and the first storage space includes multiple storage sub-spaces. The data handling unit is further configured to perform the following operations to carry the sub-matrix to be processed of the matrix to be processed to the first storage space: Moving a target submatrix to be processed from among the plurality of submatrices to be processed to the storage subspace; The subsequent sub-matrix to be processed of the target sub-matrix to be processed is moved to an available storage sub-space among the plurality of storage sub-spaces.

19. The device according to claim 18, wherein: The calculation module includes a calculation unit and a transposition unit. The calculation unit is further configured to: synchronously with the data handling unit handling the subsequent sub-matrix to be processed, perform matrix operations using the calculation unit according to the first target sub-matrix of the first matrix and the target sub-matrix to be processed, to obtain a target sub-matrix to be transposed among the multiple sub-matrices to be transposed of the matrix to be transposed; The computing unit is further configured to: store the target to-be-transposed submatrix into one of a plurality of cache subspaces of a third cache space; The transposition unit is further configured to: transpose the target submatrix to be transposed to obtain a target result submatrix among multiple result submatrices of the result matrix; and store the target result submatrix in one of the multiple cache subspaces of the fourth cache space.

20. The device according to claim 19, wherein The calculation unit is further configured to: synchronously with the transposition unit transposing the target to-be-transposed submatrix, perform matrix operations using the calculation unit according to the first target submatrix and the to-be-processed submatrix, to obtain a to-be-transposed submatrix of the target to-be-transposed submatrix; The transposition unit is further configured to: store the target result submatrix into one of a plurality of cache subspaces of a fourth cache space, and synchronously transpose the subsequent submatrix to be transposed to obtain a subsequent result submatrix of the target result submatrix.

21. The device according to claim 18, wherein The data handling unit is further configured to perform the following operations to carry the sub-matrix to be processed of the matrix to be processed to the first storage space: In response to having performed matrix operations according to the plurality of to-be-processed sub-matrices of the target to-be-processed sub-matrix, the to-be-processed sub-matrix of the subsequent to-be-processed matrix is ​​moved from the second cache space to an available storage sub-space among the plurality of storage sub-spaces.

22. The device according to claim 13, wherein: The first matrix includes a plurality of first sub-matrices, The computing module is further configured as: In response to determining that a matrix operation has been performed based on a first target submatrix among the plurality of first submatrices and a plurality of submatrices to be processed of the matrix to be processed, a matrix operation is performed based on a first subsequent submatrix of the first target submatrix and a submatrix to be processed of the matrix to be processed to obtain the result matrix.

23. An electronic device comprising the device according to any one of claims 13 to 22.

24. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 12.

25. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 12.

26. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 12.

Images