Data processing method and device, storage medium and program product

Abstract

The embodiment of the invention provides a data processing method and device, a storage medium and a program product, and relates to the technical field of cloud computing. In the embodiment of the invention, aiming at segmentation of a matrix multiplication task, a method for heuristically determining block information is provided, corresponding block information of the matrix multiplication task in multiple matrix dimensions is generated on a first calculation unit according to matrix parameters and resource parameters of a second calculation unit, and the block information is determined according to the generated block information. According to the method and the device, the block information meeting the resource occupancy rate requirement can be determined under the condition that extra overhead is hardly introduced. According to the method and the device, the determined block information is provided for the second computing unit, so that the second computing unit can perform multi-dimensional segmentation on the matrix multiplication task according to the block information, and the segmented block tasks are executed in parallel, so that the resource occupancy rate of the second computing unit during execution of the matrix multiplication task is increased.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet, and in particular to a data processing method and device, a storage medium and a program product. BACKGROUND

[0002] Matrix multiplication is a common computing task in neural networks, which has been widely applied in various neural network models such as convolutional neural networks, text-to-image models, and large language models. Matrix multiplication can be represented by a triple {m, n, k}, where A is an m-row k-column matrix, B is a k-row n-column matrix, and the product of matrix A and matrix B is a new m-row n-column matrix C.

[0003] Matrix multiplication accounts for a large proportion of neural network computing tasks, and the speed of matrix multiplication execution will significantly affect the time consumed by the neural network training process and inference process. In order to reduce the time overhead of matrix multiplication, a larger matrix multiplication task can be divided into multiple subtasks from the m-dimension, n-dimension and k-dimension, and the multiple subtasks are executed in parallel by multiple independent computing units on a GPU (Graphics Processing Unit, GPU), thereby improving the computing efficiency.

[0004] However, in actual application, the number of subtasks divided often does not match the number of computing units included in the GPU, which will result in low GPU resource occupancy. SUMMARY

[0005] The present application provides a data processing method, device, storage medium and program product, which divides a matrix multiplication task into multiple subtasks, and matches the number of subtasks with the number of computing modules on the computing unit responsible for executing the matrix multiplication task, thereby improving the resource occupancy of the computing unit.

[0006] The present application provides a data processing method, comprising: obtaining matrix parameters corresponding to a matrix multiplication task and resource parameters of a second computing unit responsible for executing the matrix multiplication task; generating block information of a first matrix and a second matrix participating in the matrix multiplication task in the row and column dimensions respectively according to the matrix parameters and the resource parameters; and sending the block information of the first matrix and the second matrix in the row and column dimensions respectively to the second computing unit, so that the second computing unit divides the matrix multiplication task into multiple first block tasks for parallel execution to obtain a result matrix corresponding to the matrix multiplication task.

[0007] The embodiment of the present application further provides an electronic device, comprising: a first computing unit and a second computing unit; the first computing unit is configured to generate block information of a first matrix and a second matrix participating in a matrix multiplication task in row and column dimensions respectively according to matrix parameters corresponding to the matrix multiplication task and resource parameters of the second computing unit, and provide the block information to the second computing unit; and the second computing unit is configured to split the matrix multiplication task into a plurality of first sub-block tasks according to the block information, and perform the plurality of first sub-block tasks in parallel to obtain a result matrix corresponding to the matrix multiplication task.

[0008] The embodiment of the present application further provides a computing unit, which can be implemented as the first computing unit, comprising: a storage module and a processing module, the storage module is configured to store a computer program, and the processing module is coupled with the storage module and is configured to execute the computer program, so as to: acquire matrix parameters corresponding to a matrix multiplication task and acquire resource parameters of a second computing unit responsible for executing the matrix multiplication task; generate block information of a first matrix and a second matrix participating in the matrix multiplication task in row and column dimensions respectively according to the matrix parameters and the resource parameters; and the block information is used for the second computing unit to split the matrix multiplication task into a plurality of first sub-block tasks to perform the plurality of first sub-block tasks in parallel to obtain a result matrix corresponding to the matrix multiplication task.

[0009] The embodiment of the present application further provides an electronic device, comprising a memory and a processor, the memory is configured to store a computer program, and the processor is coupled with the memory and is configured to execute the computer program, so as to implement steps in the data processing method provided by the embodiment of the present application.

[0010] The embodiment of the present application further provides a computer readable storage medium storing a computer program, when the computer program is executed by a processor, the processor can implement steps in the above data processing method.

[0011] The embodiment of the present application further provides a computer program product, the computer program product comprises computer programs / instructions, when the computer programs / instructions are executed by a processor, the processor can implement steps in the above data processing method.

[0012] In the embodiments of the present application, for the splitting of the matrix multiplication task, a method for heuristically determining the block information is proposed. The first computing unit estimates the block information corresponding to the matrix multiplication task in multiple matrix dimensions according to the matrix parameters and the resource parameters of the second computing unit responsible for executing the matrix multiplication task, so as to determine the block information meeting the resource occupation requirement without introducing additional overhead. Then the determined block information is provided to the second computing unit, so that the second computing unit splits the matrix multiplication task in multiple dimensions according to the block information, and performs parallel execution on the split first block task, thereby improving the resource occupation rate of the second computing unit when executing the matrix multiplication task. BRIEF DESCRIPTION OF DRAWINGS

[0013] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of the present application and illustrate embodiments of the present application and the description thereof serve to explain the present application and do not limit the present application in any way. In the drawings:

[0014] Figure 1 A process schematic diagram of the matrix multiplication task splitting provided for an exemplary embodiment of the present application;

[0015] Figure 2 A process schematic diagram of the matrix multiplication task splitting provided for another exemplary embodiment of the present application;

[0016] Figure 3 A structure schematic diagram of an electronic device provided for another exemplary embodiment of the present application;

[0017] Figure 4a A flow schematic diagram of a data processing method provided for another exemplary embodiment of the present application;

[0018] Figure 4b A flow schematic diagram of a data processing method provided for another exemplary embodiment of the present application;

[0019] Figure 5 A structure schematic diagram of an electronic device provided for another exemplary embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described below in conjunction with the embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0021] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of countries and regions, and provide corresponding operation portal for user to choose authorization or refusal. In addition, the various models (including but not limited to language models or large models) involved in the present application are in line with relevant legal and standard regulations.

[0022] In the embodiments of the present application, the matrix multiplication task is a matrix multiplication task in the training or inference process of a neural network model, which can be but is not limited to various neural network models such as convolutional neural network, text-to-image model, large language model, etc. For the matrix multiplication task, a triple {m, n, k} can be used to represent, referring to Figure 1 , which represents the multiplication of an m-row k-column matrix and a k-row n-column matrix. Figure 1 In the embodiments of the present application, the m-row k-column matrix is referred to as the first matrix; the k-row n-column matrix is referred to as the second matrix; and the product of the first matrix and the second matrix is referred to as the result matrix.

[0023] Optionally, in the embodiments of the present application, the first matrix and the second matrix are feature maps extracted from input image information, image-text information, speech information and / or text information by the neural network model, that is, the feature maps in the embodiments of the present application can be feature maps extracted from single information (such as image information, multi-modal information (such as image-text information), speech information or text information), or feature maps extracted from multi-modal information (such as image information, image-text information, speech information and text information, or fusion information of two or more of the above information), or intermediate state feature maps obtained after various calculations on the feature maps extracted from single information, which are not limited. The image information in the embodiments of the present application can be an independent image containing static visual information, or a video frame in video data containing dynamic visual information; the image-text information refers to information form containing both image and text; the text information refers to information form mainly containing various types of text information; and the speech information refers to information form mainly containing audio or speech signals.

[0024] For convenience of description, in some embodiments of the present application, the row dimension of the first matrix or the row dimension of the result matrix is referred to as the m dimension, that is, the first split dimension; the column dimension of the second matrix or the column dimension of the result matrix is referred to as the n dimension, that is, the second split dimension; and the column dimension of the first matrix or the row dimension of the second matrix is referred to as the k dimension, that is, the third split dimension. Accordingly, the split task according to the two split dimensions of the m dimension and the n dimension is referred to as a second split task; and the split task according to the three split dimensions of the m dimension, the n dimension and the k dimension is referred to as a first split task. In terms of split granularity, the split granularity of the first split task is finer, the split granularity of the second split task is coarser, and the first split task can be regarded as a subtask of the second split task.

[0025] In the embodiments of the present application, different dimensions have respective split sizes. The split size of a dimension refers to the size of the split task of the matrix multiplication task in the dimension, that is, the size of the split task, and in the embodiments, the split tasks are also matrix multiplication tasks, and the size of the split task can be represented by the number of rows and columns. For example, as shown in FIG. 3, the m dimension and the n dimension have respective split sizes, that is, the number of rows and columns included in the split task split from the m dimension and the n dimension. Figure 1

[0026] ​In the embodiments of this application, any computing unit with computing capabilities can be used to perform the matrix multiplication task. For example, the computing unit can be, but is not limited to, a CPU (Central Processing Unit), GPU, DPU (Data Processing Unit), FPGA (Field Programmable Gate Array), NPU (Neural Processing Unit), etc. In the embodiments of this application, for ease of description and distinction, the computing unit responsible for performing the matrix multiplication task is referred to as the second computing unit. The second computing unit has multiple computing modules and can execute the computing task in parallel. For example, when the second computing unit is implemented as a CPU, the CPU can include multiple CPU cores, and one CPU core can be considered as a computing module; when the second computing unit is implemented as a GPU, the GPU can include streaming multiprocessors (SMs), and one SM can be considered as a computing module, or the arithmetic logic unit included in one SM can be considered as a computing module, without limitation; when the second computing unit is implemented as an NPU, the NPU adopts a parallel architecture and can include multiple processing elements (PEs), and one processing element can be considered as a computing module, and so on. Preferably, the second computing unit can be implemented as a GPU. As a processor specifically designed for processing graphics and computationally intensive tasks, a GPU has a large number of computing modules and high-speed cache, which can accelerate the computation of matrix multiplication tasks.

[0027] To fully utilize the computing resources on the computing unit and reduce the time overhead of matrix multiplication tasks, for example... Figure 1 The matrix multiplication task shown is divided into m-dimensional and n-dimensional blocks, and each element can correspond to a second block task. Figure 1Taking element 1 as an example, the second block task corresponding to element 1 represents the task of multiplying data block 1 corresponding to the first matrix and data block 2 corresponding to the second matrix. Data block 1 includes several rows of the first matrix; data block 2 includes several columns of the second matrix. For example, the triplet of the matrix multiplication task can be represented as {1024, 512, 1024}, that is, m = 1024, n = 512, k = 1024. Then, the matrix multiplication task can be divided into multiple second block tasks from the m-dimensional and n-dimensional dimensions. If the block size in the m-dimensional dimension is 128 and the block size in the n-dimensional dimension is 128, then the number of blocks in the m-dimensional dimension is 8 and the number of blocks in the n-dimensional dimension is 4. Thus, the number of second block tasks is 4 × 8 = 32, and the size of one second block task is 128 rows and 128 columns.

[0028] Furthermore, by utilizing the multiple computing modules included in the second computing unit, multiple second-block tasks are executed in parallel, thereby improving the resource utilization and computational efficiency of the second computing unit. During the parallel execution of multiple second-block tasks, the second computing unit includes a certain number of computing modules. These modules can serve as the basic unit for scheduling and execution, executing multiple second-block tasks corresponding to the same matrix multiplication task simultaneously, with each computing module executing one second-block task. The number of computing modules included in the second computing unit is related to its hardware architecture; for example, it may include 32 computing modules, but is not limited to this. Further, in the example above, if the matrix multiplication task is divided into 32 second-block tasks, the second computing unit can utilize 32 computing modules to execute 32 second-block tasks in parallel at once. In this case, the number of divided second-block tasks is the same as the number of computing modules on the second computing unit, which is a matching situation. Therefore, the resource utilization of the second computing unit is relatively high. In this embodiment, the resource utilization rate of the second computing unit using computing modules to execute second-block tasks is referred to as the second utilization rate.

[0029] However, in practical applications, a significant mismatch often occurs between the number of second-stage tasks and the number of computing modules contained in the second computing unit. This mismatch can occur because: the number of second-stage tasks is much smaller than the number of computing modules in the second computing unit; or the number of second-stage tasks is much larger than the number of computing modules in the second computing unit, but because the second computing unit can only execute a limited number of computing modules in parallel at a time, multiple task rounds are required to complete the execution of the second-stage tasks. Consequently, in some executions, the computing modules in the second computing unit may not be fully utilized, resulting in low resource utilization. Here, "much smaller than" or "much larger than" refers to the absolute value of the difference between the two quantities being greater than a set threshold. The following sections will describe these two scenarios separately.

[0030] In one example, the triplet of the matrix multiplication task is represented as {128, 128, 1024}, that is, m = 128, n = 128, k = 1024. It can be seen that m and n are relatively small while k is relatively large in this matrix multiplication task. If we still take the example above where the block size in the m dimension is 128 and the block size in the n dimension is 128, then the number of second block tasks will be 1. Assuming that the matrix multiplication task is executed by the second computing unit which includes 32 computing modules, the number of second block tasks is much smaller than the number of computing modules on the second computing unit (such as 32). As a result, 31 computing modules will be idle, leading to a low resource utilization rate of the second computing unit. In another example, assuming the matrix multiplication task is divided into m and n dimensions, resulting in 33 second-part tasks, the second computing unit needs to execute two task rounds to complete the execution of the 33 second-part tasks. That is, the first task round executes 32 second-part tasks, and the second task round executes 1 part task. It can be seen that the computing modules on the second computing unit cannot be fully utilized during the execution of the second task round, resulting in a low resource utilization rate.

[0031] Based on this, in the embodiments of this application, such as Figure 2 As shown, based on the segmentation of the matrix multiplication task in m and n dimensions, it is further segmented in k dimensions, further dividing the second block task corresponding to element 1 into several smaller-granularity first block tasks. The several first block tasks corresponding to each element 1 are executed in parallel using different computing modules on the second computing unit, thereby improving resource utilization. In this process, with... Figure 2Taking any first block task corresponding to element 1 as an example, this first block task reads data block 3 from the first matrix and data block 4 from the second matrix, and performs multiplication to obtain the execution result of this first block task. Then, the execution results of these first block tasks are reduced and summed along the k-dimensional plane to obtain the execution result of the second block task corresponding to element 1. Similar to the second block task, the number of first block tasks also affects the resource utilization of the second computing unit. If the number of first block tasks is severely mismatched with the number of computing modules on the second computing unit, it will lead to a low resource utilization of the second computing unit.

[0032] Furthermore, in this embodiment, a given matrix multiplication task can be divided into different numbers of first block tasks. These different numbers of first block tasks are then executed cyclically using the computing modules on the second computing unit to obtain the resource utilization rates corresponding to each number of first block tasks. The number of first block tasks whose resource utilization rates meet the requirements is then selected as the number of blocks in the k-dimensional region for subsequent neural network training and inference. The resource utilization rate of the second computing unit executing the first block tasks is called the first resource utilization rate. However, since different numbers of first block tasks are run alternately on the second computing unit to obtain the corresponding first resource utilization rates, continuous testing and optimization are required to determine a suitable number of first block tasks, which also incurs a certain time overhead. Additionally, this process needs to be repeated for different matrix multiplication tasks, and as the number of matrix multiplication tasks increases, a considerable amount of time overhead will also be generated.

[0033] Furthermore, partitioning along the third partitioning dimension (k-dimensional) incurs additional reduction overhead. The larger the number of blocks along the third partitioning dimension, the greater the reduction overhead. When the reduction overhead outweighs the performance gains from increasing resource utilization, it results in a negative outcome.

[0034] Therefore, in this embodiment of the application, a heuristic method for determining block information is proposed for the segmentation of matrix multiplication tasks. This method is executed by a first computing unit. Based on the matrix parameters and the resource parameters of a second computing unit, the first computing unit estimates the block information corresponding to the matrix multiplication task in multiple dimensions. The block information that meets the resource utilization requirements is determined with almost no additional overhead. The determined block information is then provided to a second computing unit so that it can segment the matrix multiplication task in multiple dimensions according to the block information and execute the segmented first block task in parallel, thereby improving the first resource utilization rate.

[0035] In addition, to address the reduction overhead of matrix multiplication tasks, a first resource utilization threshold is set. This threshold is used to constrain the number of first-stage tasks that are split into segments, seeking a balance between reduction overhead and performance gains from splitting, and avoiding excessive reduction overhead.

[0036] The following detailed description, in conjunction with the accompanying drawings, illustrates the heuristic method for determining block information provided in the embodiments of this application.

[0037] Figure 3 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this application. For example... Figure 3 As shown, the electronic device 300 includes a first computing unit 301 and a second computing unit 302. The first computing unit 301 generates block information for the first and second matrices in the row and column dimensions, respectively, for the second computing unit 302. This block information allows the second computing unit 302 to divide the matrix multiplication task into subdivisions, ensuring that the number of subdivisions meets the resource utilization requirements of the second computing unit 302 during execution. This allows the second computing unit to maintain a high resource utilization rate (i.e., a high first resource utilization rate) when executing the matrix multiplication task, even when multiple subdivisions are computed in parallel.

[0038] There are no restrictions on the internal implementation structure of the first computing unit 301 and the second computing unit 302. Figure 3 An example is given, in which the first computing unit 301 includes an acquisition module 303, a generation module 304 and a sending module 305; the second computing unit 302 includes multiple computing modules 306 capable of independently executing block tasks. Figure 3 The internal implementation structure of the first computing unit 301 and the second computing unit 302 shown can be either a software structure or a hardware structure.

[0039] In this embodiment, the first computing unit can be various hardware resources with certain computing resources, such as, but not limited to, CPU, GPU, FPGA, or DPU. The first computing unit can obtain the matrix parameters corresponding to the matrix multiplication task. The source of the matrix parameters is not limited; for example, it can be obtained after the first computing unit communicates with the second computing unit; or it can be manually set. Optionally, the acquisition module 303 of the first computing unit 301 is used to acquire the matrix parameters, which include the number of rows and columns of the first matrix participating in the matrix multiplication task, and the number of rows and columns of the second matrix. Wherein, the multiplication of the first matrix and the second matrix is ​​the result matrix. The matrix multiplication task can be represented as a triple {m, n, k}, then the first matrix has m rows and k columns; the second matrix has k rows and n columns, and the result matrix has m rows and n columns.

[0040] The matrix multiplication task can be divided into multiple first block tasks to utilize the computing units on the second computing unit for parallel execution, thereby improving the resource utilization and computational efficiency of the second computing unit. In this embodiment, dividing the matrix multiplication task into multiple first block tasks depends on the block information of the first matrix and the second matrix in the row and column dimensions. This block information is specifically generated by the first computing unit 301 based on the matrix parameters corresponding to the matrix multiplication task and the resource parameters of the second computing unit.

[0041] Furthermore, the first computing unit can also obtain resource parameters of the second computing unit responsible for performing matrix multiplication tasks. Optionally, the acquisition module 303 of the first computing unit 301 is used to acquire resource parameters, which include the number of computing modules on the second computing unit capable of independently executing various block tasks, such as 32, 64, etc., depending on the hardware architecture of the second computing unit.

[0042] The first computing unit generates block information for the first and second matrices in the row and column dimensions based on matrix parameters and resource parameters. Optionally, the generation module 304 of the first computing unit is used to generate the block information. Further, the sending module 305 of the first computing unit provides the block information for the first and second matrices in the row and column dimensions to the second computing unit, so that the second computing unit 302 can segment the first and second matrices in their respective row and column dimensions based on this information. The block information for the first and second matrices in the row and column dimensions includes block information for the first matrix in its row dimension (m dimension), block information for the first matrix in its column dimension (k dimension), block information for the second matrix in its row dimension (k dimension), and block information for the second matrix in its column dimension (n dimension). It should be noted that block information in the same dimension is the same, i.e., the block information for the first matrix in its column dimension (k dimension) and the block information for the second matrix in its row dimension (k dimension). For ease of description, the row and column dimensions of the first and second matrices will be referred to as "each dimension," meaning each dimension of the first and second matrices. Since "each dimension" includes both the row and column dimensions of the first and second matrices, the row dimension corresponds to the number of rows, and the column dimension corresponds to the number of columns.

[0043] In this embodiment, the block information for each dimension may include the size of the blocks to be divided in each dimension; or, it may be the number of blocks to be divided in each dimension, without limitation. When the block information is implemented as block size, the block information for each dimension may also include the number of rows or columns in that dimension. Specifically, when the dimension is a row dimension, the block information also includes the number of rows, or when the dimension is a column dimension, the block information also includes the number of columns. In this way, the number of blocks in the row dimension can be calculated based on the block size and the number of rows, or the number of blocks in the column dimension can be calculated based on the block size and the number of columns.

[0044] Furthermore, the block information of the first matrix and the second matrix in the row and column dimensions is sent to the second computing unit so that the computing module on the second computing unit can split the matrix multiplication task into multiple first block tasks for parallel execution based on the block information of the first matrix and the second matrix in the row and column dimensions.

[0045] In this embodiment, the second computing unit includes multiple computing modules. These modules can determine the data blocks that each first block task should read from the first matrix based on the block information of the first matrix in the row and column dimensions. Similarly, they can determine the data blocks that each first block task should read from the second matrix based on the block information of the second matrix in the row and column dimensions. The implementation method of having the computing modules on the second computing unit execute multiple first block tasks in parallel is not limited. Optionally, the second computing unit includes multiple computing modules, each with a number, ordered from smallest to largest. The multiple computing modules can determine the data blocks that each first block task should read from the first matrix based on the block information of the first matrix in the row and column dimensions, and similarly, they can determine the data blocks that each first block task should read from the second matrix based on the block information of the second matrix in the row and column dimensions. Then, they sequentially read the corresponding data blocks from the first and second matrices according to the order of their numbers and perform multiplication operations to complete the execution of the corresponding first block task. Each data block refers to a small matrix in the first or second matrix, from row x1 to row x2 and from column y1 to column y2. x1 and x2 are determined based on the block information of the first or second matrix in their respective row dimensions. For the first matrix, x1 ≤ x2 and x2 ≤ m; for the second matrix, x1 ≤ x2 and x2 ≤ k. y1 and y2 are determined based on the block information of the first or second matrix in their respective column dimensions. For the first matrix, y1 ≤ y2 and y2 ≤ k; for the second matrix, y1 ≤ y2 and y2 ≤ n.

[0046] Alternatively, the second computing unit may include multiple computing modules, from which one can be selected as the master computing module. This master computing module is responsible for determining the data blocks that each first block task should read from the first matrix based on the block information of the first matrix in the row and column dimensions, and determining the data blocks that each first block task should read from the second matrix based on the block information of the second matrix in the row and column dimensions. Then, the reading tasks for the data blocks corresponding to each first block task are assigned to other computing modules. The other computing modules read the required data blocks from the first and second matrices according to the assigned data reading tasks, execute the data blocks, and perform multiplication operations to complete the execution of the corresponding first block task. In this optional method, the first block task handled by the computing module is not necessarily related to the number of the computing module. Regardless of the execution method described above, the execution result of the matrix multiplication task is finally obtained by reducing and summing the execution results of multiple first block tasks.

[0047] In this embodiment, a heuristic method for determining block information is proposed for the segmentation of matrix multiplication tasks. A first computing unit estimates the block information corresponding to the matrix multiplication task across multiple dimensions based on matrix parameters and resource parameters of a second computing unit, and provides this block information to the second computing unit. This differs from the second computing unit determining the final block information by actually executing the segmented tasks under various block information. This approach determines block information that meets resource utilization requirements with almost no additional overhead. The determined block information is then provided to the second computing unit, allowing it to segment the matrix multiplication task across multiple dimensions and execute the segmented tasks in parallel, thereby improving the resource utilization of the second computing unit when executing matrix multiplication tasks. Furthermore, optimizing the matrix multiplication task improves the user experience of neural network model training or inference services and reduces the deployment cost of training or inference services.

[0048] In an optional embodiment, when the first computing unit generates the block information of the first matrix and the second matrix in the row and column dimensions for participating in the matrix multiplication task according to the matrix parameters and resource parameters, it includes: generating the block information of the first matrix and the second matrix in the row and column dimensions according to the matrix parameters and resource parameters, with the goal of the computing module on the second computing unit executing the first block task to meet the first resource occupancy rate requirement.

[0049] Optionally, the matrix parameters include the number of rows and columns of the first matrix and the second matrix, and the resource parameters include the number of computing modules on the second computing unit capable of independently executing the block task. Then, based on the number of rows in the first matrix and the number of columns in the second matrix, block information on the first and second partitioning dimensions is generated. The first partitioning dimension is the row dimension of the result matrix corresponding to the first matrix or the matrix multiplication task, and the second partitioning dimension is the column dimension of the result matrix corresponding to the second matrix or the matrix multiplication task. Based on the number of computing modules, the block information on the first and second partitioning dimensions, and with the goal of ensuring that the computing modules on the second computing unit execute the first block task to meet the first resource utilization requirement, block information on the third partitioning dimension is generated. The third partitioning dimension is either the column dimension of the first matrix or the row dimension of the second matrix. In this embodiment, block information on the first and second segmentation dimensions is generated first. Based on this, block information on the third segmentation dimension is generated with the goal of ensuring that the computing modules on the second computing unit execute the first segmented task to meet the first resource utilization requirement. The block information on the third segmentation dimension is used to ensure that the number of first sub-tasks is matched with the number of computing modules in the second computing unit. There is no need to limit the generation method of block information on the first and second segmentation dimensions, which provides greater flexibility and helps to reduce the amount of computation.

[0050] In this embodiment, the method of generating block information based on the number of rows in the first matrix and the number of columns in the second matrix is ​​not limited. Two generation methods are provided below, but are not limited to these.

[0051] In one optional implementation, when generating block information for the first and second segmentation dimensions based on the number of rows in the first matrix and the number of columns in the second matrix, the number of blocks in the first segmentation dimension is generated based on the number of rows in the first matrix and the block size in the first segmentation dimension. This method is simple, easy to implement, and highly efficient. The ratio of the number of rows or columns corresponding to each dimension to its corresponding block size is the number of blocks in that dimension. For example, the ratio of the number of rows in the first segmentation dimension to the block size in the first segmentation dimension is the number of blocks in the first segmentation dimension, and this is used as the block information for the first segmentation dimension. Similarly, the number of blocks in the second segmentation dimension is generated based on the number of columns in the second matrix and the block size in the second segmentation dimension, and this is used as the block information for the second segmentation dimension. The ratio of the number of columns in the second segmentation dimension to the block size in the second segmentation dimension is the number of blocks in the second segmentation dimension. In this optional embodiment, the number of blocks is used as the block information for illustration, but it is not limited to this. For example, the block information can also be the block size. The number of blocks can also be obtained based on the number of rows or columns corresponding to each dimension and the block size corresponding to that dimension.

[0052] In this embodiment, the method for determining the block size on the first and second partitioning dimensions is not limited. For example, the block size may differ in different application scenarios and / or hardware environments, and a suitable block size can be determined based on rules of thumb. Optionally, determining the block size based on rules of thumb includes: conducting multiple tests and optimizations for specific application scenarios and / or hardware characteristics to determine a suitable block size, and then presetting the suitable block size in the matrix parameters for reading the block size on different partitioning dimensions from the matrix parameters. For another example, the block size on the first and second partitioning dimensions is related to the number of rows or columns of their respective dimensions. Therefore, the correspondence between the number of rows in the first matrix and the block size, and the correspondence between the number of columns in the second matrix and the block size can be queried based on the number of rows in the first matrix and the number of columns in the second matrix, respectively, thereby determining the block size on the first and second partitioning dimensions. Optionally, the above correspondence can be determined based on a segmentation strategy. In one example, the segmentation strategy may include obtaining a block size that is divisible by the number of rows in the first matrix and the number of columns in the second matrix. Further, the segmentation strategy may also include: when the number of rows in the first matrix and the number of columns in the second matrix are relatively small, the block size can be relatively small, for example, the block size for the first segmentation dimension and the second segmentation dimension can be 32; when the number of rows in the first matrix and the number of columns in the second matrix are relatively large, the block size can be relatively large, for example, the block size for the first segmentation dimension and the second segmentation dimension can be 128. It should be understood that the block size values ​​here are merely examples and do not constitute a limitation on the implementation.

[0053] In another optional implementation, when generating block information in the first and second partitioning dimensions based on the number of rows in the first matrix and the number of columns in the second matrix, the process includes: generating block information in the first and second partitioning dimensions based on the number of computing modules, the number of rows in the first matrix, and the number of columns in the second matrix, with the goal of ensuring that the computing modules on the second computing unit execute the second partitioning task to meet the second resource utilization requirement; wherein, the second partitioning task refers to the partitioning task obtained by splitting the matrix multiplication task according to the block information in the first and second partitioning dimensions. In this embodiment, the block information can be the block size in the corresponding partitioning dimension; or, it can be the number of blocks in the corresponding partitioning dimension, without limitation. This implementation aims to ensure that the computing modules on the second computing unit execute the second partitioning task to meet the second resource utilization requirement, and can generate block information in the first and second partitioning dimensions more accurately when considering the second resource utilization. In addition, since the block information in the first and second segmentation dimensions has already taken into account the second resource utilization rate, the generation process of the block information in the third segmentation dimension is more likely to meet the requirements of the first resource utilization rate, which helps to simplify the generation process of the block information in the third segmentation dimension.

[0054] In this embodiment, when generating block information in the first and second segmentation dimensions, multiple candidate block information in the first and second segmentation dimensions are determined. These candidate block information are then combined with the multiple candidate block information in the second segmentation dimension to obtain multiple block information combinations. Different combinations of block information are used to divide the matrix multiplication task into different numbers of second block tasks. Here, one combination includes one candidate block information in the first and second segmentation dimensions, and different combinations include multiple combinations formed between each candidate block information in the first and second segmentation dimensions.

[0055] Optionally, determining multiple candidate block information on the first segmentation dimension includes: obtaining the initial and upper limits of the block information on the first segmentation dimension from the matrix parameters; and generating multiple candidate block information based on a preset step size, initial value, and upper limit. For example, when the block information is implemented as a number of blocks, taking an initial value of 1, a step size of 1, and an upper limit of 8 for the candidate block information on the first segmentation dimension as an example, the candidate block information on the first segmentation dimension would be blocks numbered 1-8. The upper limit of the block information can be determined empirically, and different application scenarios and / or hardware characteristics may have different upper limits. The method for determining multiple candidate block information on the second segmentation dimension can be referenced to the method for determining multiple candidate block information on the first segmentation dimension, and will not be repeated here. It should be noted that the initial value, upper limit, and step size of the block information on the second segmentation dimension can be the same as or different from those on the first segmentation dimension.

[0056] Furthermore, for any combination, based on the number of computing modules and the number of second block tasks corresponding to any combination, the resource utilization rate of the second computing unit under any combination is estimated; from multiple combinations, a target combination whose resource utilization rate meets the second resource utilization rate requirement is selected, and the candidate block information in the target combination is used as the block information on the first segmentation dimension and the block information on the second segmentation dimension, respectively. The method for estimating the resource utilization rate of the second computing unit under any combination can be found in the following embodiments, which describe the process of "in the process of generating block information on the third segmentation dimension, for any candidate block information on the third segmentation dimension, based on the number of first block tasks and the number of computing modules corresponding to any candidate block information, the resource utilization rate of the second computing unit under that candidate block information"; this will not be detailed here.

[0057] Having obtained the first segmentation dimension and the block information on the first segmentation dimension, block information on the third segmentation dimension is further generated. Optionally, before generating the block information on the third segmentation dimension, a first judgment operation on whether to generate the block information on the third segmentation dimension can also be performed. The first judgment operation is used to determine whether the first matrix and the second matrix involved in the current matrix multiplication task meet the requirements of a narrow and long matrix. If the requirements of a narrow and long matrix are met, the block information on the third segmentation dimension is generated so that the second computing unit can segment the matrix multiplication task on the third segmentation dimension according to the block information. Here, a narrow and long matrix refers to a matrix where the first ratio of the number of rows of the first matrix to the number of columns of the second matrix is ​​less than a first ratio threshold, and / or the second ratio of the number of columns of the second matrix to the number of rows of the second matrix is ​​less than a second ratio threshold. For example, in the above embodiment, the matrix multiplication task is represented as a triple {m,n,k}. In this case, a narrow and long matrix can be represented as m / k being less than the first ratio threshold (e.g., 0.5), and / or n / k being less than the second ratio threshold (e.g., 0.5).

[0058] Based on this, before generating the block information on the third segmentation dimension, the first judgment operation includes: calculating a first ratio of the number of rows in the first matrix to the number of rows in the second matrix, and a second ratio of the number of columns in the second matrix to the number of rows in the second matrix; if the first ratio is less than a first ratio threshold and / or the second ratio is less than a second ratio threshold, the operation of generating the block information on the third segmentation dimension is performed. If the first ratio is greater than or equal to the first ratio threshold and the second ratio is greater than or equal to the second ratio threshold, it indicates that neither the first matrix nor the second matrix is ​​a narrow and long matrix, and the operation of generating the block information on the third segmentation dimension can be omitted, that is, no segmentation is performed on the third segmentation dimension, and the block information on the first and second segmentation dimensions is directly provided to the second computing unit, so that the second computing unit can divide the matrix multiplication task into multiple second block tasks for parallel execution based on the block information on the first and second segmentation dimensions.

[0059] Optionally, if the first ratio is less than a first ratio threshold and / or the second ratio is less than a second ratio threshold, a second judgment operation can be performed. Specifically, if the matrix multiplication task is divided into a second target number of second block tasks along the first and second segmentation dimensions, the resource utilization rate of the second computing unit executing the second target number of second block tasks using the computing module is estimated, and it is determined whether this resource utilization rate is less than a second resource utilization rate threshold. If the resource utilization rate is less than the second resource utilization rate threshold, that is, if the second block task is used as the final segmentation granularity of the matrix multiplication task, then the resource utilization rate of the second computing unit executing the second block task will not meet the second resource utilization rate requirement. In this case, an operation to generate block information along a third segmentation dimension can be performed, so that the second computing unit can divide the second block task along the third segmentation dimension according to this block information to obtain multiple first block tasks with smaller granularity, thereby improving the resource utilization rate of the second computing unit. The second resource utilization rate threshold can be, but is not limited to, 0.5, 0.65, 0.78, etc.

[0060] Based on the above, when the first ratio is less than the first ratio threshold and / or the second ratio is less than the second ratio threshold, the number of second block tasks to be divided into the matrix multiplication task is determined according to the block information on the first and second partitioning dimensions, and this number is taken as the second target number. Based on the second target number and the number of computing modules, the resource utilization rate of the second computing unit using the computing modules to execute the second target number of second block tasks is estimated. When the resource utilization rate is less than the second resource utilization rate threshold, the operation of generating block information on the third partitioning dimension is performed. Details regarding the operation of generating block information on the third partitioning dimension can be found in subsequent embodiments and will not be repeated here. Further optionally, when the resource utilization rate is greater than or equal to the second resource utilization rate threshold, the operation of generating block information on the third partitioning dimension may not be performed; that is, partitioning is not performed on the third partitioning dimension, and the block information on the first and second partitioning dimensions is directly provided to the second computing unit so that the second computing unit can divide the matrix multiplication task into multiple second block tasks for parallel execution based on the block information on the first and second partitioning dimensions.

[0061] In this embodiment, either the first judgment operation or the second judgment operation can be selected; alternatively, the first judgment operation and the second judgment operation can be used in combination, without limitation. Furthermore, there is no limitation on the execution order of the first judgment operation and the second judgment operation.

[0062] Further, the first computing unit performs the operation of generating block information on the third segmentation dimension. In an optional embodiment, when generating block information on the third segmentation dimension based on the number of computing units, the block information on the first segmentation dimension, and the block information on the second segmentation dimension, with the goal of the computing modules on the second computing unit first executing the block task to meet the first resource utilization requirement, the process includes: determining multiple candidate block information on the third segmentation dimension; combining the candidate block information on different third segmentation dimensions with the block information on the first and second segmentation dimensions to divide the matrix multiplication task into different numbers of first block tasks; then, for any candidate block information on the third segmentation dimension, calculating the resource utilization rate of the second computing unit under any candidate block information based on the number of first block tasks corresponding to any candidate block information and the number of computing modules; and selecting candidate block information whose resource utilization rate meets the first resource utilization requirement from the multiple candidate block information on the third segmentation dimension as the block information on the third segmentation dimension. The first segment task has a smaller granularity than the second segment task, and the first segment task can be obtained by further segmenting the second segment task in the third segmentation dimension.

[0063] Optionally, when determining multiple candidate block information in the third segmentation dimension, the initial and upper limits of the block information in the third segmentation dimension can be obtained from the matrix parameters. Based on the preset step size, initial value, and upper limit, multiple candidate block information are generated. For example, when the block information is implemented as a number of blocks, the initial value of the candidate block information can be 1, the step size is 1, and the upper limit is 8. Then, the multiple candidate block information in the third segmentation dimension includes (1, 2, 3, 4, 5, 6, 7, 8). The upper limit can be determined empirically, and different application scenarios and / or hardware characteristics may result in different upper limits. Alternatively, multiple candidate block information in the third segmentation dimension can be generated based on the number of columns in the first matrix or the number of rows in the second matrix, combined with the preset correspondence between the number of rows and columns and the upper limit of the block information. According to the matrix multiplication principle, if the number of columns in the first matrix is ​​the same as the number of rows in the second matrix, then the number of columns in the first matrix and the number of rows in the second matrix are collectively referred to as the number of rows and columns. Different numbers of rows and columns can have different upper limits for block information. Given the number of rows and columns, the corresponding upper limit for block information can be determined. Based on the upper limit for block information, multiple candidate block information can be determined. For example, if the upper limit for block information is 16, then any natural number value less than or equal to 16 can be used as candidate block information. This natural number value does not include 0.

[0064] Furthermore, for any candidate block information in the third segmentation dimension, based on the number of first block tasks and the number of computing modules on the second computing unit corresponding to any candidate block information, the resource utilization rate of the second computing unit under any candidate block information is estimated. From multiple candidate block information in the third segmentation dimension, the candidate block information whose resource utilization rate meets the first resource utilization rate requirement is selected as the block information in the third segmentation dimension. Here, the resource utilization rate of the second computing unit under any candidate block information refers to the estimated resource utilization rate of the computing modules on the second computing unit when performing the matrix multiplication task to obtain the first block task according to that candidate block information, given the number of first block tasks and the number of first block tasks.

[0065] Optionally, when there are multiple candidate block information pieces that meet the first resource utilization requirement, the selection of candidate block information is based on the different implementation methods of the candidate block information, using the smaller number of first block tasks derived from the candidate block information as the selection criterion. For example, if the candidate block information is implemented as the number of blocks, a smaller number of blocks can be selected, such as the smallest, resulting in fewer first block tasks and lower resource overhead for subsequent reduction and summation. As another example, if the block information is implemented as the block size, the block size and the number of blocks are inversely proportional. Therefore, in this case, a larger block size can be selected, such as the largest, resulting in fewer first block tasks and lower resource overhead for subsequent reduction and summation.

[0066] In this embodiment, during the process of estimating the resource utilization rate of the second computing unit under any candidate block information, multiple candidate block information on the third segmentation dimension can be traversed. For the current candidate block information traversed, the number of first block tasks to be split into the matrix multiplication task is determined by combining the block information on the first segmentation dimension and the block information on the second segmentation dimension, and this number is used as the first target quantity. Specifically, the number of blocks in each dimension can be determined based on the block information of each dimension, and the number of blocks in each dimension is multiplied to obtain the number of first block tasks, which is used as the first target quantity. In this embodiment, each dimension includes the first segmentation dimension, the second segmentation dimension, and the third segmentation dimension.

[0067] Furthermore, based on the first target quantity and the number of computing modules, the number of task rounds required for the second computing unit to execute the first target quantity of first block tasks is generated. As mentioned above, the second computing unit may include a certain number of computing modules capable of independently executing block tasks. Therefore, the ratio of the first target quantity to the number of computing modules independently executed in a single task round of the second computing unit, rounded up, yields the number of task rounds for the second computing unit. For example, if the first target quantity is 48 and the second computing unit includes 32 computing modules, then 48 / 32, rounded up, indicates that two task rounds are needed to complete the execution of the first target quantity of first block tasks. Here, the symbol " / " represents division.

[0068] Furthermore, based on the first target quantity, the number of computing modules, and the task rounds, the resource utilization rate of the second computing unit under the current candidate block information is estimated. Specifically, the total number of computing modules in the second computing unit participating in the execution of multiple first block tasks can be calculated by multiplying the number of computing modules in the second computing unit by the number of task rounds. Then, the ratio of the first target quantity to the total number of modules participating in the execution of first block tasks is the first resource utilization rate. Referring to the previous example, when the first target quantity is 48 and the second computing unit includes 32 computing modules, two task rounds are required. Therefore, the number of computing modules participating in the matrix multiplication task is 2 × 32 = 64. Thus, the first resource utilization rate is 48 / 64 = 0.75.

[0069] Optionally, if the resource utilization rate of the second computing unit under the current candidate block information is less than the first resource utilization rate threshold, the next candidate block information is traversed. If the resource utilization rate of the second computing unit under the current candidate block information is greater than or equal to the first resource utilization rate threshold, it means that the resource utilization rate of the second computing unit is already high and meets the resource utilization rate requirement. In order to reduce the reduction overhead caused by subtask splitting, the traversal can be stopped. Further, optionally, if the block information is implemented as the number of blocks, if different numbers of blocks correspond to the same resource utilization rate, the smaller number of blocks is selected first to reduce the reduction overhead.

[0070] Furthermore, the block information that meets the first resource utilization requirement is provided to the second computing unit, so that the computing module on the second computing unit can split the matrix multiplication task into multiple first block tasks for parallel execution based on the block information of the first matrix and the second matrix in the row and column dimensions.

[0071] In this embodiment, the entire block information generation process is executed on the first computing unit, with resource utilization as the target. It can heuristically generate block information across various dimensions with almost no additional overhead, and it has no requirements on the type or model of the second computing unit. Furthermore, it is applicable to evidence multiplication tasks between various matrix sizes, demonstrating universality. Moreover, by setting certain conditions, acceleration can be limited to multiplication tasks between narrow and long matrices, avoiding negative performance gains in conventional matrix multiplication scenarios. This application's technical solution allows for on-demand selection. Furthermore, the proposed technical solution can also solve the problem of closed-source code, enabling various cloud vendors to have more flexible and controllable acceleration capabilities when facing richer matrix multiplication fusion scenarios.

[0072] The data processing method is described in the following embodiments. Detailed information about the data processing method can be found in the foregoing embodiments and will not be repeated in subsequent embodiments.

[0073] Figure 4a This is a flowchart illustrating a data processing method provided as another exemplary embodiment of this application. The method is applied to a first computing unit, such as... Figure 4a As shown, the method includes:

[0074] S401a: Obtain the matrix parameters corresponding to the matrix multiplication task and the resource parameters of the second computing unit responsible for executing the matrix multiplication task;

[0075] S402a: Based on the matrix parameters and the resource parameters, generate the block information of the first matrix and the second matrix participating in the matrix multiplication task in the row and column dimensions, respectively;

[0076] S403a: The block information of the first matrix and the second matrix in the row and column dimensions is sent to the second computing unit so that the second computing unit can divide the matrix multiplication task into multiple first block tasks for parallel execution to obtain the result matrix corresponding to the matrix multiplication task.

[0077] In an optional embodiment, based on the matrix parameters and the resource parameters, generating block information in the row and column dimensions of the first and second matrices participating in the matrix multiplication task includes:

[0078] Based on the matrix parameters and the resource parameters, with the goal of the computing unit on the second computing unit executing the first block task to meet the first resource utilization requirement, the block information of the first matrix and the second matrix in the row and column dimensions is generated respectively.

[0079] In this embodiment, a heuristic method for determining block information is proposed for the segmentation of matrix multiplication tasks. Based on the matrix parameters of the matrix multiplication task and the resource parameters of the second computing unit executing the matrix multiplication task, the first computing unit generates block information corresponding to the matrix multiplication task across multiple matrix dimensions. This method can determine block information that meets resource utilization requirements with almost no additional overhead. The determined block information is then provided to the second computing unit, which performs multi-dimensional segmentation of the matrix multiplication task based on the block information and executes the segmented block tasks in parallel, thereby improving the resource utilization of the second computing unit when executing the matrix multiplication task.

[0080] about Figure 4a For detailed descriptions of each step and its technical effects, please refer to [link / reference]. Figure 4b The embodiments shown are not described in detail here.

[0081] Figure 4b This is a flowchart illustrating a data processing method provided as another exemplary embodiment of this application. The method is applied to a first computing unit, such as... Figure 4b As shown, the method includes:

[0082] S401b: Obtain the matrix parameters corresponding to the matrix multiplication task. The matrix parameters include the number of rows and columns of the first and second matrices involved in the matrix multiplication task. The matrix multiplication task can be divided into multiple first block tasks.

[0083] S402b: Obtain the resource parameters of the second computing unit responsible for performing the matrix multiplication task. The resource parameters include the number of computing modules on the second computing unit that can independently execute the block task.

[0084] S403b: Based on the matrix parameters and resource parameters, with the goal of the computing module on the second computing unit executing the first block task to meet the first resource utilization rate requirement, generate the block information of the first matrix and the second matrix in the row and column dimensions, respectively.

[0085] S404b: The block information of the first matrix and the second matrix in the row and column dimensions is sent to the second computing unit so that the computing module on the second computing unit can split the matrix multiplication task into multiple first block tasks for parallel execution based on the block information of the first matrix and the second matrix in the row and column dimensions.

[0086] In an optional embodiment, the matrix parameters include the number of rows and columns of the first matrix and the second matrix, and the resource parameters include the number of computing modules on the second computing unit capable of independently executing block tasks. Based on the matrix parameters and the resource parameters, and with the goal of the computing modules on the second computing unit executing the first block task to meet a first resource utilization requirement, block information in the row and column dimensions of the first matrix and the second matrix is ​​generated, including: generating block information in a first segmentation dimension and block information in a second segmentation dimension based on the number of rows of the first matrix and the number of columns of the second matrix; wherein, the first segmentation dimension is the row dimension of the first matrix or the result matrix corresponding to the matrix multiplication task, and the second segmentation dimension is the column dimension of the second matrix or the result matrix corresponding to the matrix multiplication task; based on the number of computing modules, the block information in the first segmentation dimension, and the block information in the second segmentation dimension, and with the goal of the computing modules on the second computing unit executing the first block task to meet the first resource utilization requirement, the block information in the third segmentation dimension is generated; wherein, the third segmentation dimension is the column dimension of the first matrix or the row dimension of the second matrix.

[0087] In an optional embodiment, based on the number of computing modules, the block information on the first segmentation dimension, and the block information on the second segmentation dimension, and with the objective of the computing modules on the second computing unit first executing the block task to meet the first resource utilization requirement, the block information on the third segmentation dimension is determined, including: determining multiple candidate block information on the third segmentation dimension; combining different candidate block information with the block information on the first segmentation dimension and the block information on the second segmentation dimension to divide the matrix multiplication task into different numbers of first block tasks; for any candidate block information on the third segmentation dimension, estimating the resource utilization rate of the second computing unit under any candidate block information based on the number of first block tasks corresponding to any candidate block information and the number of computing modules; and selecting candidate block information whose resource utilization rate meets the first resource utilization requirement from the multiple candidate block information on the third segmentation dimension as the block information on the third segmentation dimension.

[0088] In one optional embodiment, determining multiple candidate block information on the third segmentation dimension includes: obtaining initial and upper limit values ​​of block information on the third segmentation dimension from the matrix parameters; generating the multiple candidate block information according to a preset step size, the initial and upper limit values; or, generating the multiple candidate block information according to the number of columns of the first matrix or the number of rows of the second matrix, combined with a preset correspondence between the number of rows and columns and the upper limit value of the block information.

[0089] In an optional embodiment, for any candidate block information on the third segmentation dimension, the resource utilization rate of the second computing unit under any candidate block information is estimated based on the number of first block tasks corresponding to any candidate block information and the number of computing modules. This includes: traversing multiple candidate block information on the third segmentation dimension; for the current candidate block information traversed, determining the number of first block tasks to be divided into the matrix multiplication task by combining the block information on the first segmentation dimension and the block information on the second segmentation dimension, as a first target number; generating the number of task rounds required for the second computing unit to execute the first target number of first block tasks using the computing modules based on the first target number and the number of computing modules; and estimating the resource utilization rate of the second computing unit under the current candidate block information based on the first target number, the number of computing modules, and the task rounds.

[0090] In an optional embodiment, the method further includes: if the resource utilization rate of the second computing unit under the current candidate block information is less than the first resource utilization rate threshold, continuing to traverse the next candidate block information; and stopping the traversal if the resource utilization rate of the second computing unit under the current candidate block information is greater than or equal to the first resource utilization rate threshold.

[0091] In one optional embodiment, generating block information in the first segmentation dimension and block information in the second segmentation dimension based on the number of rows of the first matrix and the number of columns of the second matrix includes: generating the number of blocks in the first segmentation dimension based on the number of rows of the first matrix and the block size in the first segmentation dimension, as the block information in the first segmentation dimension; and generating the number of blocks in the second segmentation dimension based on the number of columns of the second matrix and the block size in the second segmentation dimension, as the block information in the second segmentation dimension.

[0092] In an optional embodiment, the method further includes: obtaining the block size on the first segmentation dimension and the block size on the second segmentation dimension from the matrix parameters; or, querying the correspondence between the number of rows and columns and the block size according to the number of rows in the first matrix and the number of columns in the second matrix, respectively, to determine the block size on the first segmentation dimension and the block size on the second segmentation dimension.

[0093] In an optional embodiment, generating block information in a first and second partitioning dimension based on the number of rows in the first matrix and the number of columns in the second matrix includes: generating block information in the first and second partitioning dimensions based on the number of computing modules, the number of rows in the first matrix, and the number of columns in the second matrix, with the goal of the computing modules on the second computing unit executing a second partitioning task to meet a second resource utilization requirement; wherein, the second partitioning task refers to a partitioning task obtained by partitioning the matrix multiplication task based on the block information in the first and second partitioning dimensions.

[0094] In an optional embodiment, based on the number of computing modules, the number of rows in the first matrix, and the number of columns in the second matrix, and with the goal of ensuring that the computing modules on the second computing unit perform the second segmented task to meet the second resource utilization requirement, segmentation information on the first segmentation dimension and segmentation information on the second segmentation dimension are generated. This includes: determining multiple candidate segmentation information on the first segmentation dimension and multiple candidate segmentation information on the second segmentation dimension; wherein, different combinations between the multiple candidate segmentation information on the first segmentation dimension and the multiple candidate segmentation information on the second segmentation dimension are used to segment the matrix multiplication task into different numbers of second segmented tasks; for any combination, estimating the resource utilization rate of the second computing unit under any combination based on the number of computing modules and the number of second segmented tasks corresponding to any combination; selecting a target combination from the multiple combinations whose resource utilization rate meets the second resource utilization requirement, and using the candidate segmentation information in the target combination as the segmentation information on the first segmentation dimension and the segmentation information on the second segmentation dimension, respectively.

[0095] In an optional embodiment, before generating the block information on the third segmentation dimension, the method further includes: calculating a first ratio of the number of rows in the first matrix to the number of rows in the second matrix, and a second ratio of the number of columns in the second matrix to the number of rows in the second matrix; if the first ratio is less than a first ratio threshold and / or the second ratio is less than a second ratio threshold, performing an operation to generate the block information on the third segmentation dimension based on the number of computing modules, the block information on the first segmentation dimension, and the block information on the second segmentation dimension, with the goal of the computing modules on the second computing unit executing the first segmentation task to meet the first resource utilization requirement.

[0096] In an optional embodiment, when the first ratio is less than a first ratio threshold and / or the second ratio is less than a second ratio threshold, an operation is performed to generate the block information in the third segmentation dimension based on the number of computing modules, the block information in the first segmentation dimension, and the block information in the second segmentation dimension, with the goal of the computing modules on the second computing unit executing the first block task to meet the first resource utilization requirement. This operation includes: determining the number of second block tasks to be divided into the matrix multiplication task based on the block information in the first segmentation dimension and the block information in the second segmentation dimension, as a second target number; estimating the resource utilization rate of the second computing unit using computing modules to execute the second target number of second block tasks based on the second target number and the number of computing modules; and, when the resource utilization rate is less than the second resource utilization rate threshold, generating the block information in the third segmentation dimension based on the number of computing modules, the block information in the first segmentation dimension, and the block information in the second segmentation dimension, with the goal of the computing modules on the second computing unit executing the first block task to meet the first resource utilization requirement.

[0097] In an optional embodiment, the matrix multiplication task is a matrix multiplication task in the training or inference process of a neural network model, and the first matrix and the second matrix are feature maps extracted by the neural network model from input image information, image and text information, speech information and / or text information.

[0098] The detailed implementation methods and beneficial effects of each step in this embodiment have been described in detail in the foregoing embodiments, and will not be elaborated here.

[0099] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 401a to 403a can be device A; or the execution subject of steps 401a and 402a can be device A, and the execution subject of step 403a can be device B; and so on.

[0100] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 401a, 402a, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0101] Figure 5 This is a schematic diagram of the structure of an electronic device provided as another exemplary embodiment of this application. For example... Figure 5 As shown, the device includes: a memory 54 and a processor 55.

[0102] Memory 54 is used to store computer programs and can be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device, contact data, phone book data, messages, pictures, videos, etc.

[0103] The processor 55, coupled to the memory 54, is used to execute the computer program in the memory 54 for: obtaining the matrix parameters corresponding to the matrix multiplication task and the resource parameter calculation unit module of the second computing unit responsible for executing the matrix multiplication task; generating block information of the first matrix and the second matrix in the row and column dimensions according to the matrix parameters and the resource parameters; and sending the block information of the first matrix and the second matrix in the row and column dimensions to the second computing unit so that the second computing unit module can divide the matrix multiplication task into multiple first block tasks for parallel execution to obtain the result matrix corresponding to the matrix multiplication task.

[0104] In an optional embodiment, the processor 55 generates block information in the row and column dimensions of the first matrix and the second matrix participating in the matrix multiplication task based on the matrix parameters and the resource parameters. Specifically, it generates block information in the row and column dimensions of the first matrix and the second matrix based on the matrix parameters and the resource parameters, with the goal of the computing module on the second computing unit executing the first block task to meet the first resource utilization requirement.

[0105] In an optional embodiment, the matrix parameters include the number of rows and columns of the first matrix and the second matrix, and the resource parameters include the number of computing modules on the second computing unit capable of independently executing block tasks; when the processor 55 generates block information of the first matrix and the second matrix in the row and column dimensions according to the matrix parameters and the resource parameters, with the goal of the computing modules on the second computing unit executing the first block task to meet the first resource utilization requirement, it is specifically used to: generate block information in the first segmentation dimension and block information in the second segmentation dimension according to the number of rows of the first matrix and the number of columns of the second matrix; wherein, the first segmentation dimension is the row dimension of the first matrix or the result matrix corresponding to the matrix multiplication task, and the second segmentation dimension is the column dimension of the second matrix or the result matrix corresponding to the matrix multiplication task; and generate block information in the third segmentation dimension according to the number of computing modules, the block information in the first segmentation dimension and the block information in the second segmentation dimension, with the goal of the computing modules on the second computing unit executing the first block task to meet the first resource utilization requirement; wherein, the third segmentation dimension is the column dimension of the first matrix or the row dimension of the second matrix.

[0106] In an optional embodiment, when the processor 55 determines the block information on the third segmentation dimension based on the number of computing modules, the block information on the first segmentation dimension, and the block information on the second segmentation dimension, with the goal of the computing modules on the second computing unit executing the first block task to meet the first resource utilization requirement, the processor 55 specifically performs the following: determining multiple candidate block information on the third segmentation dimension; combining different candidate block information with the block information on the first segmentation dimension and the block information on the second segmentation dimension to divide the matrix multiplication task into different numbers of first block tasks; for any candidate block information on the third segmentation dimension, estimating the resource utilization rate of the second computing unit under any candidate block information based on the number of first block tasks corresponding to any candidate block information and the number of computing modules; and selecting candidate block information whose resource utilization rate meets the first resource utilization requirement from the multiple candidate block information on the third segmentation dimension as the block information on the third segmentation dimension.

[0107] In an optional embodiment, when determining multiple candidate block information on the third segmentation dimension, the processor 55 is specifically configured to: obtain the initial value and upper limit value of the block information on the third segmentation dimension from the matrix parameters; generate the multiple candidate block information according to a preset step size, the initial value and the upper limit value; or, generate the multiple candidate block information according to the number of columns of the first matrix or the number of rows of the second matrix, combined with the preset correspondence between the number of rows and columns and the upper limit value of the block information.

[0108] In an optional embodiment, when the processor 55 estimates the resource utilization rate of the second computing unit under any candidate block information based on the number of first block tasks corresponding to any candidate block information and the number of computing modules, it specifically performs the following steps: traversing multiple candidate block information on the third segmentation dimension; for the current candidate block information, combining the block information on the first segmentation dimension and the block information on the second segmentation dimension, determining the number of first block tasks to be divided into the matrix multiplication task as a first target number; generating the number of task rounds required for the second computing unit to execute the first target number of first block tasks using the computing modules based on the first target number and the number of computing modules; and estimating the resource utilization rate of the second computing unit under the current candidate block information based on the first target number, the number of computing modules, and the task rounds.

[0109] In an optional embodiment, the processor 55 is further configured to: continue traversing the next candidate block information when the resource utilization rate of the second computing unit under the current candidate block information is less than the first resource utilization rate threshold; and stop traversing when the resource utilization rate of the second computing unit under the current candidate block information is greater than or equal to the first resource utilization rate threshold.

[0110] In an optional embodiment, when the processor 55 generates block information in the first segmentation dimension and block information in the second segmentation dimension based on the number of rows of the first matrix and the number of columns of the second matrix, it is specifically used to: generate the number of blocks in the first segmentation dimension based on the number of rows of the first matrix and the block size in the first segmentation dimension, as the block information in the first segmentation dimension; and generate the number of blocks in the second segmentation dimension based on the number of columns of the second matrix and the block size in the second segmentation dimension, as the block information in the second segmentation dimension.

[0111] In an optional embodiment, the processor 55 is further configured to: obtain the block size on the first segmentation dimension and the block size on the second segmentation dimension from the matrix parameters; or, query the correspondence between the number of rows and columns of the first matrix and the block size respectively, and determine the block size on the first segmentation dimension and the block size on the second segmentation dimension based on the number of rows and columns of the second matrix.

[0112] In an optional embodiment, when the processor 55 generates block information in the first and second partitioning dimensions based on the number of rows in the first matrix and the number of columns in the second matrix, it is specifically configured to: generate block information in the first and second partitioning dimensions based on the number of computing modules, the number of rows in the first matrix, and the number of columns in the second matrix, with the goal of the computing modules on the second computing unit executing a second partitioning task to meet a second resource utilization requirement; wherein, the second partitioning task refers to a partitioning task obtained by partitioning the matrix multiplication task based on the block information in the first and second partitioning dimensions.

[0113] In an optional embodiment, when the processor 55 generates block information on the first segmentation dimension and block information on the second segmentation dimension based on the number of computing modules, the number of rows in the first matrix, and the number of columns in the second matrix, with the goal of the computing modules on the second computing unit executing the second block task to meet the second resource utilization requirement, the processor 55 specifically performs the following: determining multiple candidate block information on the first segmentation dimension and multiple candidate block information on the second segmentation dimension; wherein, different combinations between the multiple candidate block information on the first segmentation dimension and the multiple candidate block information on the second segmentation dimension are used to divide the matrix multiplication task into different numbers of second block tasks; for any combination, estimating the resource utilization rate of the second computing unit under any combination based on the number of computing modules and the number of second block tasks corresponding to any combination; selecting a target combination from the multiple combinations whose resource utilization rate meets the second resource utilization requirement, and using the candidate block information in the target combination as the block information on the first segmentation dimension and the block information on the second segmentation dimension, respectively.

[0114] In an optional embodiment, before generating the block information on the third segmentation dimension, the processor 55 is further configured to: calculate a first ratio of the number of rows in the first matrix to the number of rows in the second matrix, and a second ratio of the number of columns in the second matrix to the number of rows in the second matrix; if the first ratio is less than a first ratio threshold and / or the second ratio is less than a second ratio threshold, perform an operation to generate the block information on the third segmentation dimension based on the number of computing modules, the block information on the first segmentation dimension, and the block information on the second segmentation dimension, with the goal of the computing modules on the second computing unit performing the first segmentation task to meet the first resource utilization requirement.

[0115] In an optional embodiment, when the first ratio is less than a first ratio threshold and / or the second ratio is less than a second ratio threshold, the processor 55, when performing the operation of generating the block information in the third segmentation dimension based on the number of computing modules, the block information in the first segmentation dimension, and the block information in the second segmentation dimension, with the goal of having the computing modules on the second computing unit execute the first block task to meet the first resource utilization requirement, specifically performs the following: determining the number of second block tasks to be divided into the matrix multiplication task based on the block information in the first segmentation dimension and the block information in the second segmentation dimension, as a second target number; estimating the resource utilization rate of the second computing unit using the computing modules to execute the second target number of second block tasks based on the second target number and the number of computing modules; and, when the resource utilization rate is less than the second resource utilization threshold, performing the operation of generating the block information in the third segmentation dimension based on the number of computing modules, the block information in the first segmentation dimension, and the block information in the second segmentation dimension, with the goal of having the computing modules on the second computing unit execute the first block task to meet the first resource utilization requirement.

[0116] In an optional embodiment, the matrix multiplication task is a matrix multiplication task in the training or inference process of a neural network model, and the first matrix and the second matrix are feature maps extracted by the neural network model from input image information, image and text information, speech information and / or text information.

[0117] Furthermore, such as Figure 5 As shown, the electronic device also includes other components such as a communication component 56, a display 57, a power supply component 58, and an audio component 59. Figure 5 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 5 The components shown. Additionally... Figure 5 The components within the dashed box are optional, not mandatory, and their specific requirements depend on the product form of the electronic device. The electronic device in this embodiment can be a terminal device such as a desktop computer, laptop computer, smartphone, or IoT device, or a server-side device such as a conventional server, cloud server, or server array. If the electronic device in this embodiment is a terminal device such as a desktop computer, laptop computer, or smartphone, it may include... Figure 5 The components within the dashed box; if the electronic device in this embodiment is implemented as a conventional server, cloud server, or server array, etc., it may be omitted. Figure 5 The component within the dashed box.

[0118] Accordingly, this application embodiment also provides a computing unit, which can be implemented as the first computing unit in the above embodiments, including: a storage module and a processing module. The storage module is used to store a computer program, and the processing module is coupled to the storage module and is used to execute the computer program to: obtain matrix parameters corresponding to the matrix multiplication task and obtain resource parameters of the second computing unit responsible for executing the matrix multiplication task; generate block information in the row and column dimensions of the first matrix and the second matrix participating in the matrix multiplication task according to the matrix parameters and the resource parameters; the block information is used for the second computing unit to divide the matrix multiplication task into multiple first block tasks to execute the multiple first block tasks in parallel to obtain the result matrix corresponding to the matrix multiplication task.

[0119] For a detailed description of the various operations performed by the computing unit and the beneficial effects produced, please refer to the foregoing embodiments, which will not be repeated here.

[0120] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program / instruction, which, when executed by a processor, enables the processor to implement the steps in the methods described above.

[0121] This application also provides a computer program product, which includes a computer program / instructions that, when executed by a processor, enable the processor to implement the steps described in the above method embodiments.

[0122] The aforementioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0123] The aforementioned communication components are configured to facilitate wired or wireless communication between the device containing the communication components and other devices. The device containing the communication components can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication components receive broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication components also include a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wide Band (UWB), Bluetooth (BT), and other technologies.

[0124] The aforementioned display includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen can be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation.

[0125] The aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.

[0126] The aforementioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0127] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, compact disc read-only memory (CD-ROM), optical storage, etc.) containing computer-usable program code.

[0128] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0129] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0130] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0131] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interfaces, network interfaces, and memory.

[0132] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0133] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0134] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0135] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A data processing method, characterized by, The method applied on a first computing unit comprises: obtaining matrix parameters corresponding to a matrix multiplication task and resource parameters of a second computing unit responsible for executing the matrix multiplication task; generating, according to the matrix parameters and the resource parameters, block information of a first matrix and a second matrix participating in the matrix multiplication task in row and column dimensions respectively; downloading the block information of the first matrix and the second matrix in row and column dimensions respectively to the second computing unit, so that the second computing unit splits the matrix multiplication task into a plurality of first block tasks for parallel execution to obtain a result matrix corresponding to the matrix multiplication task.

2. The method of claim 1, wherein, According to the matrix parameters and the resource parameters, generating block information of a first matrix and a second matrix participating in the matrix multiplication task in row and column dimensions respectively, comprises: According to the matrix parameters and the resource parameters, generating block information of the first matrix and the second matrix in row and column dimensions respectively, with the goal that a computing unit on the second computing unit executing a first block task meets a first resource occupancy requirement.

3. The method of claim 2, wherein, The matrix parameters include the number of rows and columns of the first matrix and the second matrix, and the resource parameters include the number of computing modules on the second computing unit that can independently execute a block task; According to the matrix parameters and the resource parameters, generating block information of the first matrix and the second matrix in row and column dimensions respectively, with the goal that a computing module on the second computing unit executing a first block task meets a first resource occupancy requirement, comprises: According to the number of rows of the first matrix and the number of columns of the second matrix, generating block information in a first split dimension and block information in a second split dimension; Wherein, the first split dimension is the row dimension of the first matrix or the result matrix corresponding to the matrix multiplication task, and the second split dimension is the column dimension of the second matrix or the result matrix corresponding to the matrix multiplication task; According to the number of computing modules, the block information in the first split dimension and the block information in the second split dimension, generating block information in a third split dimension, with the goal that a computing module on the second computing unit executing a first block task meets a first resource occupancy requirement; Wherein, the third split dimension is the column dimension of the first matrix or the row dimension of the second matrix.

4. The method of claim 3, wherein, According to the number of computing modules, the block information in the first split dimension and the block information in the second split dimension, generating block information in a third split dimension, with the goal that a computing module on the second computing unit executing a first block task meets a first resource occupancy requirement, comprises: Determining a plurality of candidate block information in the third split dimension, and different candidate block information is combined with the block information in the first split dimension and the block information in the second split dimension to split the matrix multiplication task into a different number of first block tasks; For any candidate block information on the third split dimension, according to the number of first block tasks corresponding to the any candidate block information and the number of computing modules, the resource occupation rate of the second computing unit under the any candidate block information is estimated; From the plurality of candidate block information on the third split dimension, the candidate block information whose resource occupation rate meets the first resource occupation rate requirement is selected as the block information on the third split dimension.

5. The method of claim 4, wherein, For any candidate block information on the third split dimension, according to the number of first block tasks corresponding to the any candidate block information and the number of computing modules, the resource occupation rate of the second computing unit under the any candidate block information is estimated, including: Iterating the plurality of candidate block information on the third split dimension; For the current candidate block information iterated, the number of first block tasks obtained by splitting the matrix multiplication task is determined as a first target number in combination with the block information on the first split dimension and the block information on the second split dimension; According to the first target number and the number of computing modules, the task rounds required for the second computing unit to execute the first target number of first block tasks using computing modules are generated; According to the first target number, the number of computing modules and the task rounds, the resource occupation rate of the second computing unit under the current candidate block information is estimated.

6. The method of claim 5, wherein, Further comprising: In the case that the resource occupation rate of the second computing unit under the current candidate block information is less than the first resource occupation rate threshold, the next candidate block information is iterated; In the case that the resource occupation rate of the second computing unit under the current candidate block information is greater than or equal to the first resource occupation rate threshold, the iteration is stopped.

7. The method according to any one of claims 3-6, characterized in that, According to the number of rows of the first matrix and the number of columns of the second matrix, the block information on the first split dimension and the block information on the second split dimension are generated, including: According to the number of rows of the first matrix and the block size on the first split dimension, the number of blocks on the first split dimension is generated as the block information on the first split dimension; According to the number of columns of the second matrix and the block size on the second split dimension, the number of blocks on the second split dimension is generated as the block information on the second split dimension.

8. The method according to any one of claims 3-6, characterized in that, According to the number of rows of the first matrix and the number of columns of the second matrix, the block information on the first split dimension and the block information on the second split dimension are generated, including: According to the number of computing modules, the number of rows of the first matrix and the number of columns of the second matrix, the block information on the first split dimension and the block information on the second split dimension are generated with the goal that the second block tasks executed by the computing modules on the second computing unit meet the second resource occupation rate requirement; Wherein, the second block task refers to the block task obtained by splitting the matrix multiplication task according to the block information on the first split dimension and the block information on the second split dimension.

9. The method according to any one of claims 3-6, characterized in that, Before generating the block information on the third split dimension, further comprising: a first ratio of a number of rows of the first matrix to a number of rows of the second matrix, and a second ratio of a number of columns of the second matrix to the number of rows of the second matrix; in a case where the first ratio is less than a first proportion threshold and / or the second ratio is less than a second proportion threshold, performing an operation of generating block information in the third split dimension according to the number of computing modules, the block information in the first split dimension, and the block information in the second split dimension, so that the computing modules on the second computing unit perform the first block tasks to meet a first resource occupancy requirement.

10. The method of claim 9, wherein, in a case where the first ratio is less than a first proportion threshold and / or the second ratio is less than a second proportion threshold, performing an operation of generating block information in the third split dimension according to the number of computing modules, the block information in the first split dimension, and the block information in the second split dimension, so that the computing modules on the second computing unit perform the first block tasks to meet a first resource occupancy requirement, including: determining, according to the block information in the first split dimension and the block information in the second split dimension, a number of second block tasks into which the matrix multiplication task is split, as a second target number; estimating, according to the second target number and the number of computing modules, a resource occupancy of the second computing unit in performing the second target number of second block tasks by using the computing modules; in a case where the resource occupancy is less than a second resource occupancy threshold, performing an operation of generating block information in the third split dimension according to the number of computing modules, the block information in the first split dimension, and the block information in the second split dimension, so that the computing modules on the second computing unit perform the first block tasks to meet a first resource occupancy requirement.

11. The method according to any one of claims 1-6 and 10, characterized in that, The matrix multiplication task is a matrix multiplication task in a training or inference process of a neural network model, and the first matrix and the second matrix are feature maps extracted by the neural network model from input image information, image-text information, voice information, and / or text information.

12. An electronic device, comprising: including: a first computing unit and a second computing unit; The first computing unit is configured to generate block information of a first matrix and a second matrix in a row and column dimension according to matrix parameters corresponding to a matrix multiplication task and resource parameters of the second computing unit, and provide the block information to the second computing unit. The second computing unit is configured to split the matrix multiplication task into a plurality of first block tasks according to the block information, and perform the plurality of first block tasks in parallel to obtain a result matrix corresponding to the matrix multiplication task.

13. The apparatus of claim 12, wherein, The first computing unit includes: an obtaining module configured to obtain matrix parameters corresponding to a matrix multiplication task and resource parameters of the second computing unit; generating, by a generating module, block information of the first matrix and the second matrix in row and column dimensions respectively, according to the matrix parameters and the resource parameters, with a target that the first block tasks performed by the computing modules on the second computing unit satisfy the first resource occupancy requirement; sending, by a sending module, the block information to the second computing unit.

14. The apparatus of claim 13, wherein, The matrix parameters include row and column numbers of the first matrix and the second matrix, and the resource parameters include a number of computing modules capable of independently performing block tasks on the second computing unit. The generating module is specifically configured to: generate block information in a first split dimension and block information in a second split dimension according to the number of rows of the first matrix and the number of columns of the second matrix; The first split dimension is a row dimension of the first matrix or a result matrix corresponding to the matrix multiplication task, and the second split dimension is a column dimension of the second matrix or a result matrix corresponding to the matrix multiplication task. generate block information in a third split dimension according to the number of computing modules, the block information in the first split dimension, and the block information in the second split dimension, with a target that the first block tasks performed by the computing modules on the second computing unit satisfy the first resource occupancy requirement; The third split dimension is a column dimension of the first matrix or a row dimension of the second matrix.

15. A computing unit, implementable as a first computing unit, characterized in that, comprise: a storage module and a processing module, the storage module is configured to store a computer program, and the processing module is coupled to the storage module and configured to execute the computer program, so as to: obtain matrix parameters corresponding to a matrix multiplication task and obtain resource parameters of a second computing unit responsible for executing the matrix multiplication task; generate block information of a first matrix and a second matrix participating in the matrix multiplication task in row and column dimensions respectively according to the matrix parameters and the resource parameters; The block information is used for the second computing unit to split the matrix multiplication task into a plurality of first block tasks to perform the plurality of first block tasks in parallel to obtain a result matrix corresponding to the matrix multiplication task.

16. An electronic device, comprising: comprise a memory and a processor, the memory is configured to store a computer program, and the processor is coupled to the memory and configured to execute the computer program, so as to implement steps in the method of any one of claims 1-11.

17. A computer readable storage medium storing computer programs / instructions, characterized in that, When the computer program / instruction is executed by the processor, the processor can implement the steps in the method of any one of claims 1-11.

18. A computer program product, characterised in that, comprise: a computer program / instruction, when the computer program / instruction is executed by the processor, the processor can implement the steps in the method of any one of claims 1-11.

Images