Computing task optimization method and device, equipment, medium and program product

Abstract

The invention relates to the technical field of artificial intelligence, and provides a calculation task optimization method, device and equipment, a medium and a program product.The method comprises the steps that the slice size of each dimension is determined based on the shape of input data of a calculation task and the number of calculation units on calculation hardware; based on the slice size of each dimension, segmenting the calculation task into a plurality of calculation sub-tasks, and allocating the plurality of calculation sub-tasks to each calculation unit; on the basis of the target dimension in the dimensions and the plurality of calculation sub-tasks, dividing the calculation units into different calculation clusters; based on any computing unit in any computing cluster, computing the computing subtask to obtain a computing result of any computing unit, and accumulating the computing result of any computing unit and computing results of other computing units in any computing cluster to obtain an output result. The method can improve the utilization rate of hardware resources, accelerate the calculation process, improve the calculation efficiency and realize calculation task optimization.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a computing task optimization method, device, equipment, medium and program product. Background Art

[0002] In the field of modern computing, especially in the fields of deep learning, big data analysis, and high-performance computing, the complexity of computing tasks and the size of data are increasing, and the demand for computing efficiency is becoming more and more urgent. In order to meet this challenge, general-purpose accelerators, such as GPU (Graphics Processing Unit) and FPGA (Field Programmable Gate Array), have become the first choice for processing large-scale computing tasks due to their powerful parallel processing capabilities. These accelerators use a large number of computing units (CU, such as computing units in GPUs or logic blocks in FPGAs) to perform multiple computing tasks simultaneously, thereby significantly improving computing speed.

[0003] In deep learning scenarios, matrix multiplication and convolution operations are the most basic and frequent computing operations. In order to maximize the computing efficiency of the accelerator, the input matrix or input feature map and convolution kernel are usually split to adapt to the parallel processing architecture of the computing unit inside the accelerator. This splitting strategy allows each computing unit to process part of the data independently, thereby achieving parallel execution of computing tasks. However, for input data with smaller dimensions (such as input matrix or input feature map), it is impossible to start enough computing units for parallel processing, resulting in low utilization of computing units, which in turn causes a waste of hardware resources and a decrease in overall computing efficiency. Summary of the invention

[0004] The present invention provides a computing task optimization method, device, equipment, medium and program product to solve the defects of low computing unit utilization, hardware resource waste and low computing efficiency in the related art when processing computing tasks with small input data dimensions.

[0005] The present invention provides a computing task optimization method, comprising: Determine the slice size of each dimension of the input data based on the shape of the input data of the computing task and the number of computing units on the computing hardware; Based on the slice size of each dimension, the computing task is divided into a plurality of computing subtasks, and the plurality of computing subtasks are allocated to each computing unit; Based on the target dimension in each dimension and the multiple computing subtasks, the computing units are divided into different computing clusters; Based on any computing unit in any computing cluster, the computing subtask is calculated to obtain a computing result of the any computing unit, and the computing result of the any computing unit is accumulated with the computing results of other computing units in the any computing cluster to obtain an output result, and the computing units in the any computing cluster share the computing results.

[0006] According to a computing task optimization method provided by the present invention, the dimensions further include a first dimension and a second dimension, and the computing task is divided into a plurality of computing subtasks based on the slice size of the dimensions, including: Based on the slice size of the first dimension and the slice size of the second dimension, dividing the computing task into a plurality of initial subtasks; Based on the slice size of the target dimension, each initial subtask is divided to obtain the multiple computing subtasks.

[0007] According to a computing task optimization method provided by the present invention, the computing units are divided into different computing clusters based on the target dimension in each dimension and the multiple computing subtasks, including: In the target dimension, computing units corresponding to all computing subtasks required to compute the same initial subtask are allocated to the same computing cluster.

[0008] According to a computing task optimization method provided by the present invention, the computing subtask is calculated based on any computing unit in any computing cluster to obtain the computing result of any computing unit, and the computing result of any computing unit is accumulated with the computing results of other computing units in any computing cluster to obtain an output result, including: In any one of the computing clusters, computing the computing subtask based on any one of the computing units, and obtaining a computing result of any one of the computing units; Broadcasting the calculation result of any one of the computing units to the shared memory of other computing units in any one of the computing clusters, so that the other computing units read the calculation result of any one of the computing units from the corresponding shared memory; Based on any one of the computing units, reading the calculation result of any one of the computing units from the local register, and reading the calculation results of each of the other computing units from the shared memory corresponding to any one of the computing units; The calculation result of any one of the calculation units is accumulated with the calculation results of the other calculation units to obtain the output result.

[0009] According to a computing task optimization method provided by the present invention, broadcasting the computing result of any computing unit to the shared memory of other computing units in any computing cluster includes: Based on the number of computing units in any computing cluster, evenly split the computing result of any computing unit to obtain multiple computing sub-results; Any calculation sub-result is stored in the local register of any of the calculation units, and the remaining calculation sub-results are broadcast one by one to the shared memory of other calculation units in any of the calculation clusters.

[0010] According to a computing task optimization method provided by the present invention, the computing task is a matrix multiplication computing task, the input data includes a left matrix and a right matrix for matrix multiplication, and the output result is an output matrix.

[0011] According to a computing task optimization method provided by the present invention, the first dimension of the dimensions is the row dimension of the left matrix, the second dimension of the dimensions is the column dimension of the right matrix, and the target dimension of the dimensions is the intermediate dimension, which refers to the column dimension of the left matrix or the row dimension of the right matrix.

[0012] The present invention also provides a computing task optimization device, comprising: A determination unit, configured to determine a slice size of each dimension of the input data based on a shape of the input data of the computing task and a number of computing units on the computing hardware; A segmentation unit, configured to segment the computing task into a plurality of computing subtasks based on the slice size of each dimension, and assign the plurality of computing subtasks to each computing unit; A partitioning unit, configured to partition the computing units into different computing clusters based on a target dimension in each dimension and the plurality of computing subtasks; An accumulation unit is used to calculate the computing subtask based on any computing unit in any computing cluster to obtain a computing result of the any computing unit, and to accumulate the computing result of the any computing unit with the computing results of other computing units in the any computing cluster to obtain an output result. The computing units in any computing cluster share the computing results.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any one of the computing task optimization methods described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the computing task optimization method described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned computing task optimization methods.

[0016] The computing task optimization method, device, equipment, medium and program product provided by the present invention determine the slice size of each dimension based on the shape of the input data of the computing task and the number of computing units on the computing hardware, and divide the computing task according to the slice size of each dimension, so that the computing task can be assigned to different computing units in a finer granularity, thereby increasing the number of computing units started, so that more computing units can participate in the calculation at the same time, which can not only improve the utilization rate of the computing units and reduce the waste of hardware resources, but also speed up the calculation process, improve the calculation efficiency, and achieve the optimization of the computing task. In addition, by dividing the computing units into different computing clusters, the various computing units in the same computing cluster can share the calculation results and cooperate with each other, so that the calculation results of each computing unit in the computing cluster are accumulated, which can ensure that the results of each computing subtask are correctly merged into the final output, thereby ensuring the integrity and accuracy of the final output result. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of the computing task optimization method provided by the present invention; Figure 2 It is a distribution diagram of the computing units and computing clusters provided by the present invention; Figure 3 It is a schematic diagram of the result accumulation of each computing unit in the computing cluster provided by the present invention; Figure 4 It is a structural schematic diagram of the computing task optimization device provided by the present invention; Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] In order to improve computing efficiency, when executing computing tasks, general accelerators will divide computing tasks into multiple computing subtasks, and execute these computing subtasks in parallel through a large number of computing units (CUs), thereby improving computing speed. For example, when implementing matrix multiplication calculations, in order to improve computing efficiency, computing tasks will be evenly distributed to multiple CUs as much as possible for parallel processing. Here, in the matrix multiplication calculation, the computing task specifically refers to the multiplication operation of each pair of elements involved in the matrix multiplication and the subsequent addition operation, which together constitute the complete calculation process of the entire matrix multiplication. When evenly distributing computing tasks, it is essentially to distribute the computing amount (i.e., the above-mentioned multiplication and addition operations) as evenly as possible to multiple CUs for parallel processing, thereby reducing the total computing time and improving computing efficiency. It should be understood that the above-mentioned computing unit can be a CU in a GPU, and each GPU usually has multiple SPCs (Streaming Processing Clusters), and each SPC includes multiple CUs.

[0021] Specifically, in the matrix multiplication calculation, for the input matrices A (shape [M, K]) and B (shape [K, N]), the calculation task is to calculate A×B to obtain a new output matrix C (shape [M, N]), where C[i][j] (representing the element in the i-th row and j-th column of matrix C) is the sum of the product of the i-th row of matrix A and the j-th column of matrix B. Here, the input matrix A is called the left matrix, the input matrix B is called the right matrix, and the shape of the matrix refers to the number of rows and columns of the matrix. For example, the shape of matrix A is [M, K], which means that matrix A has M rows and K columns. When the calculation task is evenly distributed, it means dividing the original input matrices A and B into multiple smaller sub-matrices, and then assigning these sub-matrices to different CUs for calculation.

[0022] It can be understood that for matrix A, M is the number of rows of matrix A, also known as the row dimension, and K is the number of columns of matrix A, also known as the column dimension. When implementing matrix multiplication calculation tasks (A×B=C), in order to improve the computing unit utilization and computing efficiency of the hardware, the general accelerator will evenly distribute as many CUs as possible to the M×N elements of the output matrix C. That is, the accelerator will use multiple CUs to calculate different elements in the output matrix C at the same time, and each CU is responsible for calculating one or more elements in C. This parallel processing method can significantly improve computing efficiency.

[0023] If both the M and N dimensions of the matrix are large, it is easy to evenly distribute a large number of CUs to the dimensions of M and N. For example, suppose there is a matrix multiplication task, where the shape of matrix A is [1024, 2048] (i.e., M=1024, K=2048), and the shape of matrix B is [2048, 4096] (i.e., K=2048, N=4096), and their product C needs to be calculated, whose shape is [1024, 4096]. Specifically, the matrix C can be divided into multiple smaller sub-matrices, for example, C can be divided into 16×32 sub-matrices (each sub-matrix is ​​[64, 128]) in the M and N dimensions, and the calculation of each sub-matrix will be assigned to a CU on the accelerator. Assuming that there are 512 CUs on the accelerator, these 16×32 sub-matrices can be assigned to these CUs. Each CU will read the data blocks in the A and B matrices required by its corresponding sub-matrix, perform matrix multiplication operations on the CU, calculate the results of the sub-matrices, and write the results of the sub-matrices back to the corresponding positions of the output matrix C.

[0024] However, when either or both of the two dimensions M and N are relatively small, there will be certain defects: First, the amount of calculation for matrix multiplication directly depends on the size of the matrix dimensions M, K, and N. When M and N are small, the overall amount of calculation is also reduced accordingly. If the amount of calculation is not enough to fill all available CUs, then it is impossible to start a sufficient number of CUs to process tasks in parallel, resulting in low overall hardware utilization. Secondly, when M and N are small, the calculation area (i.e., sub-matrix) allocated to each CU is very small, that is, the amount of calculation that each CU needs to process is very small, but each CU will have certain fixed overheads when starting and executing tasks, such as initialization, data loading, synchronization, etc. When the amount of calculation to be processed is very small, these fixed overheads will occupy a large part of the total execution time, resulting in a relatively short actual calculation time of the CU, thereby reducing the utilization of the CU, and the utilization of the entire hardware accelerator will also be reduced accordingly.

[0025] For example, suppose there is a matrix multiplication task, where the shape of matrix A is [64, 128] (i.e., M=64, K=128), and the shape of matrix B is [128, 64] (i.e., K=128, N=64), and their product C needs to be calculated, whose shape is [64,64]. For such a small matrix, you can try to divide C into 4×4 sub-matrices (each sub-matrix is ​​[16, 16]), so that each CU will only be responsible for calculating a very small area. Suppose there are 512 CUs on the accelerator, but in this small matrix multiplication task, only 16 or less CUs may be started to perform the calculation task, and the remaining CUs will be idle. Even if the CU is started, its utilization rate is not high, because the calculation area is very small, and the computing power of the CU far exceeds the amount of calculation required for the sub-matrix, so most of the CU's time may be spent waiting for data or other non-computational tasks. Although hardware accelerators physically have a large amount of computing resources, these resources cannot be fully utilized due to the small matrix dimensions, resulting in low overall hardware utilization.

[0026] In this regard, the present invention provides a computing task optimization method. When the dimension of the input data of the computing task is small, the computing unit is divided in each dimension to improve the hardware utilization, thereby improving the computing efficiency and overcoming the above defects. For example, for a matrix multiplication computing task, when the input matrix of the matrix multiplication is very small (that is, the M and N dimensions are small), in order to improve the hardware utilization, when dividing the task, in addition to dividing the CU in the M and N dimensions, the CU division in the K dimension is also added to divide the computing task into a finer granularity, thereby increasing the number of CU startups and improving the utilization of the CU.

[0027] Figure 1 is a flow chart of the computing task optimization method provided by the present invention, such as Figure 1 As shown, the method includes: Step 110, determining the slice size of each dimension of the input data based on the shape of the input data of the computing task and the number of computing units on the computing hardware; It should be noted that the computing task optimization method provided by the present invention can be applied to the optimization of matrix multiplication computing tasks, and can also be applied to the optimization of convolution computing tasks, and the present invention does not specifically limit this. For matrix multiplication computing tasks, the input data includes the left matrix and the right matrix for matrix multiplication, the shape of the input data is the shape of the left matrix and the right matrix, and the dimensions of the input data include the row dimension (such as M dimension) and column dimension (such as K dimension) of the left matrix and the row dimension (such as K dimension) and column dimension (such as N dimension) of the right matrix, wherein the column dimension of the left matrix is ​​the same as the row dimension of the right matrix. For convolution computing tasks, the input data may include input feature maps and convolution kernels, the shape of the input data is the shape of the input feature map and the convolution kernel, and the dimensions of the input data may include the height, width, number of channels and other dimensions of the input data. The following will take the matrix multiplication computing task as an example to introduce the method provided by the present invention.

[0028] Specifically, before performing computing task segmentation, the slice size of each dimension of the input data can be determined based on the shape of the input data of the computing task and the number of computing units on the computing hardware. Here, computing hardware refers to the physical device that performs computing operations. For example, computing hardware can include GPU, FPGA, GPGPU (General-Purpose Graphics Processing Unit), TPU (Tensor Processing Unit), etc.

[0029] In matrix multiplication, the slice size of each dimension refers to the size of the smaller blocks (or sub-matrices) that the matrix is ​​divided into along its dimension (row or column). For example, for matrix A[M, K], it is divided into multiple sub-matrices, each of which can be of size [TILEM, TILEK], and for matrix B[K, N], it is divided into multiple sub-matrices, each of which can be of size [TILEK, TILEN], which can then be assigned to different computing units for parallel computing. Here, TILEM, TILEN, and TILEK are the slice sizes of the M, N, and K dimensions, respectively.

[0030] It is understandable that in the matrix multiplication calculation task, determining the slice size of each dimension is a key step, which directly affects the parallelization degree and calculation efficiency of the task. Specifically, the slice size of each dimension can be determined jointly according to factors such as the shape of the input matrix (that is, the number of rows and columns of each matrix), the number of CUs on the computing hardware, the memory and cache size of the CU, and the maximum utilization of a single CU, that is, the size of TILEM, TILEN, and TILEK.

[0031] Step 120, based on the slice size of each dimension, the computing task is divided into a plurality of computing subtasks, and the plurality of computing subtasks are allocated to each computing unit; Specifically, after determining the slice size of each dimension, the computing task can be divided into multiple computing subtasks. Here, computing subtasks refer to the smaller task units that are easier to execute in parallel that the original computing task is divided into. In the matrix multiplication calculation, each computing subtask can correspond to the multiplication operation of a submatrix in the original matrix.

[0032] It is understandable that for matrix multiplication calculation tasks, the relevant technology usually only divides the tasks in the M and N dimensions, which easily leads to the defects of low hardware utilization and computing efficiency when the M and N dimensions are small. To this end, the embodiment of the present invention adds K-dimensional division to overcome this defect. By dividing the computing tasks into finer granularity according to the slice size of the M, N, and K dimensions, more computing subtasks can be obtained, thereby increasing the number of CU startups, reducing the computing burden of a single CU, and improving the computing speed.

[0033] Specifically, according to the determined slice size of each dimension, the starting and ending position of each slice in the matrix can be clarified. For example, for a matrix A with a shape of [M, K], the slice size of the M dimension is TILEM, and the slice size of the K dimension is TILEK, then the matrix A can be divided into (M / TILEM)×(K / TILEK) sub-matrices; for a matrix B with a shape of [K, N], the slice size of the K dimension is TILEK, and the slice size of the N dimension is TILEN, then the matrix B can be divided into (K / TILEK)×(N / TILEN) sub-matrices; each CU is responsible for completing the calculation subtask of [TILEM, TILEK]×[TILEK, TILEN]. It should be understood that the division of the calculation task can be understood as the division of the two input matrices A and B involved in the matrix multiplication calculation. The calculation subtask usually contains the necessary input data (i.e., the data of the sub-matrices of A and B after the division), the storage location of the output data, and the calculation instruction (i.e., the specific algorithm or function call for performing the matrix multiplication).

[0034] After multiple computing subtasks are obtained, these computing subtasks can be assigned to each computing unit one by one. For example, a task scheduler can be designed to assign computing subtasks to available computing units. Here, the task scheduler is also called a scheduler, which is a key component in a computer system for automatically scheduling tasks according to specific policies and rules.

[0035] Step 130, based on the target dimension in each dimension and the plurality of computing subtasks, the computing units are divided into different computing clusters; It should be noted that due to the nature of matrix multiplication, after splitting in the K dimension, the intermediate results are calculated, and these intermediate results need to be accumulated to obtain the final matrix product result. For example, for the matrix multiplication calculation task ([M, K]×[K, N]), when only splitting in the M and N dimensions, each CU needs to calculate the product of [TILEM, K]×[K,TILEN], and the result is [TILEM, TILEN] (corresponding to the complete K dimension), which can be directly written back to the corresponding position of the output matrix C. However, after adding the K dimension split, each CU needs to calculate the product of [TILEM, TILEK]×[TILEK,TILEN]. Although the result is also [TILEM, TILEN], it only corresponds to the TILEK dimension, not the calculation result in the complete K dimension, but only the intermediate result in the TILEK dimension. These intermediate results need to be accumulated to obtain the calculation result in the complete K dimension, so that the calculation result can be written back to the output matrix C.

[0036] Specifically, the embodiment of the present invention divides all CUs that calculate the same block [TILEM, TILEN] into the same computing cluster (cluster) based on the target dimension (i.e., K dimension) and computing subtasks, so that each CU in the computing cluster can share and exchange their own computing results, so as to achieve the accumulation of computing results in K dimension. Here, the calculation result of each CU is an intermediate result corresponding to the TILEK dimension. It should be understood that a computing cluster refers to a set of multiple computing units. In the matrix multiplication calculation task, the above target dimension is the K dimension.

[0037] For example, assuming that the K dimension is 1024, the [TILEM, TILEN] block in the output matrix C can be further divided in the K dimension, for example, the K dimension is divided into 8 equal parts, each part contains 128 (that is, TILEK is 128) elements (from 0 to 127, 128 to 255, ..., 960 to 1023), and each CU processes a 128-element block in the K dimension, that is, CU0 processes an element block from 0 to 127 in the K dimension, CU1 processes an element block from 128 to 255 in the K dimension, and so on, CU7 processes an element block from 960 to 1023 in the K dimension. Since CU0 to CU7 cooperate with each other to complete the calculation of [TILEM, TILEN] in the complete K dimension, these 8 CUs are divided into one computing cluster. In this computing cluster, each CU calculates [TILEM, TILEK]×[TILEK, TILEN] in parallel on its assigned K-dimensional block. After all CUs complete the calculation, the calculation results of each CU are accumulated to obtain the result of the [TILEM, TILEN] block on the complete K dimension.

[0038] Step 140, based on any computing unit in any computing cluster, calculate the computing subtask to obtain the computing result of any computing unit, and accumulate the computing result of any computing unit with the computing results of other computing units in any computing cluster to obtain an output result, and the computing units in any computing cluster share the computing result.

[0039] Specifically, after the computing clusters are divided, the computing units in each computing cluster will execute the computing subtasks assigned to them in parallel. For example, for input matrices A and B, after being split in the M, N, and K dimensions, computing unit CU0 is responsible for calculating the product of matrix A1 (i.e., submatrix of A) and B1 (i.e., submatrix of B).

[0040] In each computing unit, the results of the calculated sub-matrix multiplication will first be locally accumulated. When all computing units have completed their respective computing tasks, the local accumulated results need to be summarized into the global results, which involves communication across computing units. Here, each computing unit can transmit data through a high-speed network, or share the calculation results in a shared memory environment. After all computing units have completed the accumulation of the calculation results, the corresponding accumulated results can be obtained, and the accumulated results are written to the corresponding positions of the output matrix C to obtain the final output results. The output result here is the output matrix containing the actual results.

[0041] It can be understood that the embodiment of the present invention reduces the need for cross-cluster communication by accumulating the results of computing subtasks within the cluster, which helps to further simplify the management and coordination process of parallel computing.

[0042] The method provided by the embodiment of the present invention determines the slice size of each dimension based on the shape of the input data of the computing task and the number of computing units on the computing hardware, and divides the computing task according to the slice size of each dimension, so that the computing task can be assigned to different computing units in a finer granularity, thereby increasing the number of computing units started, so that more computing units can participate in the calculation at the same time, which can not only improve the utilization rate of the computing units and reduce the waste of hardware resources, but also speed up the calculation process, improve the calculation efficiency, and achieve the optimization of the computing task. In addition, by dividing the computing units into different computing clusters, the various computing units in the same computing cluster can share the calculation results and cooperate with each other, so that the calculation results of each computing unit in the computing cluster are accumulated, which can ensure that the results of each computing subtask are correctly merged into the final output, thereby ensuring the integrity and accuracy of the final output result.

[0043] Based on the above embodiment, the dimensions further include a first dimension and a second dimension. Accordingly, step 120 specifically includes: Step 121, based on the slice size of the first dimension and the slice size of the second dimension, dividing the computing task into a plurality of initial subtasks; Step 122, based on the slice size of the target dimension, each initial subtask is divided to obtain the multiple computing subtasks.

[0044] It should be noted that in the matrix multiplication calculation task, the first dimension above represents the row dimension of the left matrix (i.e., the M dimension), the second dimension represents the column dimension of the right matrix (i.e., the N dimension), and the target dimension is the middle dimension (i.e., the K dimension), which represents both the column dimension of the left matrix and the row dimension of the right matrix.

[0045] Specifically, in a matrix multiplication calculation task, when the calculation task is divided according to the slice sizes of the M, N, and K dimensions, the calculation task can be first divided into multiple initial subtasks according to the slice sizes of the M and N dimensions (TILEM and TILEN), and then each initial subtask can be divided into multiple calculation subtasks according to the slice size of the K dimension (TILEK). It should be understood that for the matrix multiplication calculation task [M, K]×[K, N], the initial subtask refers to the smaller-scale matrix multiplication tasks that the original matrix multiplication calculation task [M, K]×[K, N] is initially divided into according to the slice sizes of the M and N dimensions of the matrix (TILEM and TILEN). The initial subtask is a rough division based on the external dimensions of the matrix (i.e., the M and N dimensions), while the calculation subtask is a finer-grained division based on the initial subtask according to the internal dimensions of the matrix (i.e., the K dimension) to further optimize the performance of parallel computing.

[0046] For example, for ease of understanding, assuming that the left matrix A is [4, 4] (ie, M=4, K=4), and the right matrix B is [4, 4] (ie, K=4, N=4), A and B are expressed as follows:

[0047] Then the output matrix C obtained by calculating the matrix A×B can be expressed as follows:

[0048] in, represents the first ( ) row elements and the first ( ) is the sum of the products of the corresponding elements in the columns. For example:

[0049] When matrices A and B are segmented in M ​​and N dimensions, assuming TILEM=2, TILEN=2, that is, each submatrix after segmentation of A is 2 rows and 4 columns, and each submatrix after segmentation of B is 4 rows and 2 columns, then A can be segmented into submatrix A1 and submatrix A2, and B can be segmented into submatrix B1 and B2, which are expressed as follows:

[0050] According to the above submatrices, the corresponding initial subtasks can be determined, namely A1×B1, A1×B2, A2×B1, A2×B2, where the initial subtask A1×B1 can be specifically expressed as:

[0051] It is understandable that if the task is split only in the M and N dimensions, the output matrix C1 obtained by calculating the initial subtask A1×B1 is:

[0052] Furthermore, taking the initial subtask A1×B1 as an example, the initial subtask is segmented in the K dimension. Assuming TILEK=2, that is, each submatrix after A1 segmentation is 2 rows and 2 columns, and each submatrix after B1 segmentation is 2 rows and 2 columns, then A1 can be segmented into submatrices A11 and A12, and B1 can be segmented into submatrices B11 and B12, which are respectively expressed as follows:

[0053] According to the above sub-matrices, the corresponding calculation sub-tasks can be determined. In matrix multiplication, since the elements of corresponding rows and columns are multiplied, the obtained calculation sub-tasks include A11×B11 and A12×B12.

[0054] Based on any of the above embodiments, step 130 specifically includes: In the target dimension, computing units corresponding to all computing subtasks required to compute the same initial subtask are allocated to the same computing cluster.

[0055] Specifically, after the computing subtasks are divided, the CUs corresponding to all computing subtasks [TILEM, TILEK]×[TILEK, TILEN] required to calculate the same initial subtask [TILEM, K]×[K, TILEN] in the K dimension can be allocated to the same computing cluster.

[0056] For example, for the initial subtask A1×B1 (i.e., [2, 4]×[4, 2]), the computing subtasks required to calculate the initial subtask include A11×B11 and A12×B12, and both of these computing subtasks can be expressed as [2, 2]×[2, 2]. Assuming that computing unit CU0 is used to calculate A11×B11 and CU1 is used to calculate A12×B12, CU0 and CU1 can be divided into the same computing cluster.

[0057] It is understandable that the embodiments of the present invention can dynamically adjust the slice size and the division of the computing cluster according to actual needs, so as to flexibly cope with computing tasks of different scales and complexities.

[0058] Based on any of the above embodiments, step 140 specifically includes: Step 141, in any one of the computing clusters, calculating the computing subtask based on any one of the computing units to obtain a calculation result of any one of the computing units; Step 142, broadcasting the calculation result of any one of the computing units to the shared memory of other computing units in any one of the computing clusters, so that the other computing units read the calculation result of any one of the computing units from the corresponding shared memory; Step 143, based on any one of the computing units, reading the calculation result of any one of the computing units from the local register, and reading the calculation results of the other computing units from the shared memory corresponding to the any one of the computing units; Step 144, accumulating the calculation result of any one of the calculation units and the calculation results of the other calculation units to obtain the output result.

[0059] Specifically, in each computing cluster, computing units will execute the computing subtasks assigned to them in parallel. For one computing unit, after completing the corresponding computing subtask, the computing result of the computing unit will be obtained. Subsequently, the computing unit will store the computing result in a local register and broadcast it to the shared memory of other computing units in the computing cluster, so that other computing units can read the computing result of the computing unit from their corresponding shared memory to achieve result accumulation.

[0060] It is understandable that each computing unit has its own corresponding shared memory. The shared memory of a computing unit refers to the memory area in a computing cluster that can be accessed and used by multiple computing units in the cluster. The design of shared memory is to improve the efficiency of data exchange between computing units and reduce the delay and overhead of data transmission. Through shared memory, computing units can quickly read and write data without traditional network communication methods, thereby speeding up the entire computing process.

[0061] The local register of a computing unit refers to a high-speed storage unit located inside the computing unit, which is used to temporarily store the data, instruction addresses and calculation results required during the calculation process. Compared with memory, registers have higher access speed and lower access latency.

[0062] Exemplarily, taking CU0 and CU1 divided into the same computing cluster in the above embodiment as an example, in this cluster, after CU0 executes the computing subtask A11×B11 and obtains the computing result C11, it stores the computing result C11 in the local register of CU0 and broadcasts the computing result C11 to the shared memory of the computing unit CU1; after CU1 executes the computing subtask A12×B12 and obtains the computing result C12, it stores the computing result C12 in the local register of CU1 and broadcasts it to the shared memory of the computing unit CU0.

[0063] After each computing unit completes the computation subtask in parallel, obtains the computation result and broadcasts it to the shared memory, each computing unit can read its own computation result from its own local register, and then read the computation result of other computing units by accessing its own shared memory area. After obtaining all the computation results, the computing unit performs an accumulation operation and adds these results to obtain the final output.

[0064] For example, for computing unit CU0, it will read its own calculation result C11 from the local register, and read CU1's calculation result C12 from the shared memory, and then accumulate C11 and C12. For computing unit CU1, it will also read its own calculation result C12 from the local register, and read CU0's calculation result C11 from the shared memory, and then accumulate C11 and C12.

[0065] Based on any of the above embodiments, in step 142, broadcasting the calculation result of any one of the computing units to the shared memory of other computing units in any one of the computing clusters includes: Step 1421, based on the number of computing units in any computing cluster, evenly divide the computing result of any computing unit to obtain multiple computing sub-results; Step 1422: store any calculation sub-result into the local register of any of the calculation units, and broadcast the remaining calculation sub-results one by one to the shared memory of other calculation units in any of the calculation clusters.

[0066] It should be noted that from the above accumulation process of CU0 and CU1, we can see that CU0 and CU1 are both calculating the accumulation results of C11 and C12. Obviously, each computing unit is performing repeated accumulation calculations, resulting in redundant calculations and wasting the computing resources of the cluster. Moreover, each computing unit stores all the calculation results of all other computing units, which leads to a large amount of data redundancy. This redundancy not only takes up more memory resources, but also may affect computing efficiency. However, if only one CU in the cluster is responsible for reading and accumulating all calculation results from the shared memory, while other CUs do not accumulate results, this CU will become a performance bottleneck because it needs to process much more data than other CUs.

[0067] In this regard, the present invention evenly divides the calculation results of each computing unit so that each computing unit is responsible for reading and accumulating different calculation results, thereby balancing the load and keeping each CU relatively busy, thereby improving the overall parallelism and efficiency. By allowing each CU to participate in the accumulation operation and be responsible for different parts of the accumulation calculation, it can ensure that hardware resources are more fully utilized, which helps to reduce performance losses caused by waiting or idleness.

[0068] Specifically, for the above computing unit CU0, it is responsible for executing the computing subtask A11×B11, and the obtained computing result C11 can be expressed as:

[0069] Since the number of computing units in the computing cluster is 2, the computing result C11 can be evenly divided in the row dimension or column dimension. It should be understood that when the number of computing units is large or the dimension of the computing result is relatively large, it can also be divided in both the row dimension and the column dimension. For example, for the computing result C11, the computing result C11 can be divided into and , and stores the first sub-computation result in the local register, and broadcasts the second sub-computation result to the shared memory of CU1.

[0070] For the above computing unit CU1, it is responsible for executing the computing subtask A12×B12, and the obtained computing result C12 can be expressed as:

[0071] Similarly, for the calculation result C12, the calculation result C12 can be divided into and , and broadcast the first sub-computation result to the shared memory of CU0, and store the second sub-computation result in the local register.

[0072] Subsequently, CU0 is responsible for sending the sub-calculation results and The accumulated result can be expressed as , write the accumulated result to the corresponding position of the output matrix C (i.e. and ) on the top.

[0073] CU1 is responsible for calculating the sub-results and The accumulated result can be expressed as , write the accumulated result to the corresponding position of the output matrix C (i.e. and ) on the top.

[0074] In an embodiment of the present invention, in order to make each CU participate in the cumulative calculation and improve the calculation efficiency, the calculation results are further divided when the accumulation is performed, and each CU only needs to process its own cumulative calculation. For example, after CU0 obtains the calculation result, it does not need to broadcast all the calculation results to the shared memory of CU1, but only needs to divide the calculation result and write the calculation sub-result that CU1 needs to accumulate into the shared memory of CU1. In this way, CU0 and CU1 will read the calculation sub-result given by the other party from their respective shared memories, and then accumulate it with the calculation sub-result stored in their local registers, and finally write it to the output matrix, and no further accumulation is required. It should be understood that within the cluster, when each CU performs accumulation, it only needs to perform accumulation in the K dimension, so the result after each CU accumulation can be directly written to the output matrix without further accumulation.

[0075] Based on any of the above embodiments, an embodiment of the present invention provides a method for optimizing matrix multiplication computing tasks based on the collaborative strategy cluster accumulation, which aims to improve hardware utilization, especially when the M and N dimensions of the matrix are small, through more refined task division and mutual cooperation of computing units in the computing cluster. The method specifically includes the following steps: Step S1: Dimension slicing and computing cluster division

[0076] Specifically, the slice sizes (TILEM, TILEN, TILEK) of the M, N, and K dimensions are determined based on the shape of the input matrix and hardware characteristics, such as the number of CUs on the computing hardware and the maximum utilization of a single CU. In the K dimension, all computing units that calculate the same block [TILEM, TILEN] are divided into the same cluster so that the CUs in each cluster can collaborate efficiently and share the calculation results. It should be understood that the slice size of each dimension can also take into account factors such as cache size and memory bandwidth to maximize the computing memory access ratio and parallelism.

[0077] Step S2: Matrix multiplication calculation and result broadcasting

[0078] Specifically, in each cluster, each CU parallelizes the matrix multiplication part of [TILEM, TILEK]×[TILEK, TILEN] that it is responsible for. When a CU completes its calculation task, it broadcasts the result to the shared memory of other CUs in the cluster. The shared memory here can be GSM (Group-Shared Memory). GSM refers to a memory area that can be accessed and used by multiple members (i.e., CUs) within a specific group, allowing multiple members to efficiently exchange data and communicate.

[0079] Step S3, accumulation and output

[0080] Specifically, within the cluster, each CU reads the calculation results of other CUs from the corresponding GSM and accumulates them with its own calculation results. After the accumulation is completed, the corresponding accumulated results are written back to the output matrix.

[0081] Figure 2 is a schematic diagram of the distribution of computing units and computing clusters provided by the present invention, such as Figure 2 As shown in the figure, there are 8 computing units (i.e., CU0~CU7), of which two groups of CUs are divided in the M dimension, no division in the N dimension, and 4 groups of CUs are divided in the K dimension. The ID (Identifier) ​​of CU0 and CU1 in the K dimension is 0, recorded as CUKId=0, the ID of CU2 and CU3 in the K dimension is 1, recorded as CUKId=1, the ID of CU4 and CU5 in the K dimension is 2, recorded as CUKId=2, and the ID of CU6 and CU7 in the K dimension is 3, recorded as CUKId=3. The computing area of ​​each CU in the figure is [TILEM, TILEK]×[TILEK, TILEN], and every 4 CUs calculate the same block [TILEM, TILEN], i.e., CU0, CU2, CU4 and CU6 (corresponding to Figure 2CU0 / 2 / 4 / 6 in the calculation block [TILEM, TILEN], CU1, CU3, CU5 and CU7 (corresponding to Figure 2 CU1 / 3 / 5 / 7 in the cluster calculates another block [TILEM, TILEN]. Therefore, CU0, CU2, CU4 and CU6 form a cluster, and CU1, CU3, CU5 and CU7 form a cluster.

[0082] Figure 3 is a schematic diagram of the result accumulation of each computing unit in the computing cluster provided by the present invention, such as Figure 3 As shown in the figure, taking the computing cluster composed of CU0, CU2, CU4 and CU6 as an example, after each CU completes the calculation, it divides the calculation result into 4 parts in the row dimension and / or column dimension, and each CU is only responsible for accumulating the results of 1 / 4 of the [TILEM, TILEN] matrix. By dividing the accumulation tasks, the overall computing efficiency can be greatly improved.

[0083] For example, suppose [TILEM, TILEN] can be represented as , then CU0 is only responsible for accumulating its own sub-calculation results and the sub-calculation results of CU2, CU4 and CU6 to obtain CU2 is only responsible for accumulating its own calculation results and the calculation results of CU0, CU4 and CU6 to obtain CU4 is only responsible for accumulating its own calculation results and the calculation results of CU0, CU2 and CU6 to obtain CU6 is only responsible for accumulating its own calculation results and the calculation results of CU0, CU2 and CU4 to obtain .

[0084] Specifically, after obtaining the calculation result, CU0 can divide the calculation result into 4 parts, that is, obtain 4 calculation sub-results, store the first calculation sub-result in the local register, broadcast the second calculation sub-result to the GSM of CU2, broadcast the third calculation sub-result to the GSM of CU4, and broadcast the fourth calculation sub-result to the GSM of CU6; similarly, after obtaining the calculation result, CU2 divides the calculation result into 4 calculation sub-results, broadcasts the first calculation sub-result to the GSM of CU0, stores the second calculation sub-result in the local register, broadcasts the third calculation sub-result to the GSM of CU4, and broadcasts the fourth calculation sub-result to the GSM of CU6. Similarly, CU4 broadcasts the first calculation sub-result to CU0's GSM, the second calculation sub-result to CU2's GSM, the third calculation sub-result is stored in the local register, and the fourth calculation sub-result is broadcast to CU6's GSM; CU6 broadcasts the first calculation sub-result to CU0's GSM, the second calculation sub-result to CU2's GSM, the third calculation sub-result to CU4's GSM, and the fourth calculation sub-result is stored in the local register. It should be understood that Figure 3 The TLR (Thread Local Register) in the chip refers to the register used to temporarily store data during calculation.

[0085] Subsequently, CU0, CU2, CU4 and CU6 read the calculation sub-results on their respective local registers and their respective GSMs, and accumulate these calculation sub-results to obtain the accumulation results on the complete K dimensions, but these accumulation results correspond to different M and N dimensions and can be directly written to the output matrix to obtain the final output result.

[0086] It can be understood that according to the above method, assuming that a [M, K]×[K, N] matrix multiplication calculation is performed on a computing hardware containing multiple CUs, if the CU is divided only in the M and N dimensions, the maximum number of CUs that can be started is Y. When the CU division in the K dimension is increased (TILEM, TILEN remain unchanged), the number of CUs that can be started is Z (Z≥2Y). From the perspective of a single computing hardware, the utilization of the CU can be doubled at least.

[0087] For example, assume that there are 8 CUs on the computing hardware. In the matrix multiplication operation of [128, 1024]×[1024, 128], the M and N dimensions are both 128, and the K dimension is 1024. If the utilization of the CU is maximized, TILEM and TILEN cannot be less than 64. According to this evaluation, at most 4 CUs can be started, that is, two in the M dimension and two in the N dimension. However, there are now 8 CUs on the hardware, which is equivalent to starting only half of the CUs, and the utilization is not high. If the K dimension is increased, for example, 1024 is divided into 512 and 512, then two CUs can be started in the K dimension, so that a total of 8 CUs are started. In addition, when the number of CUs on the computing hardware is larger, the K dimension can be divided more finely, thereby further increasing the number of CUs started to improve hardware utilization and computing efficiency.

[0088] Based on any of the above embodiments, Figure 4 is a schematic diagram of the structure of the computing task optimization device provided by the present invention, such as Figure 4 As shown, the device comprises: A determining unit 410, configured to determine a slice size of each dimension of the input data based on a shape of the input data of the computing task and a number of computing units on the computing hardware; A splitting unit 420, configured to split the computing task into a plurality of computing subtasks based on the slice sizes of the respective dimensions, and assign the plurality of computing subtasks to the respective computing units; A partitioning unit 430, configured to partition the computing units into different computing clusters based on a target dimension in each dimension and the plurality of computing subtasks; The accumulation unit 440 is used to calculate the computing subtask based on any computing unit in any computing cluster to obtain the computing result of any computing unit, and to accumulate the computing result of any computing unit with the computing results of other computing units in any computing cluster to obtain an output result. The computing units in any computing cluster share the computing results.

[0089] The device provided by the embodiment of the present invention determines the slice size of each dimension based on the shape of the input data of the computing task and the number of computing units on the computing hardware, and divides the computing task according to the slice size of each dimension, so that the computing task can be allocated to different computing units in a finer granularity, thereby increasing the number of computing units started, so that more computing units can participate in the calculation at the same time, which can not only improve the utilization rate of the computing units and reduce the waste of hardware resources, but also speed up the calculation process, improve the calculation efficiency, and achieve the optimization of the computing task. In addition, by dividing the computing units into different computing clusters, the various computing units in the same computing cluster can share the calculation results and cooperate with each other, so that the calculation results of each computing unit in the computing cluster are accumulated, which can ensure that the results of each computing subtask are correctly merged into the final output, thereby ensuring the integrity and accuracy of the final output result.

[0090] Based on any of the above embodiments, the dimensions further include a first dimension and a second dimension. Accordingly, the segmentation unit 420 is specifically configured to: Based on the slice size of the first dimension and the slice size of the second dimension, dividing the computing task into a plurality of initial subtasks; Based on the slice size of the target dimension, each initial subtask is divided to obtain the multiple computing subtasks.

[0091] Based on any of the above embodiments, the dividing unit 430 is specifically configured to: In the target dimension, computing units corresponding to all computing subtasks required to compute the same initial subtask are allocated to the same computing cluster.

[0092] Based on any of the above embodiments, the accumulating unit 440 includes: A computing subunit, configured to compute the computing subtask based on any computing unit in any computing cluster to obtain a computing result of any computing unit; A broadcast subunit, used for broadcasting the calculation result of any one of the calculation units to the shared memory of other calculation units in any one of the calculation clusters, so that the other calculation units read the calculation result of any one of the calculation units from the corresponding shared memory; A reading subunit, used to read the calculation result of any one of the calculation units from a local register based on the any one of the calculation units, and read the calculation results of the other calculation units from a shared memory corresponding to the any one of the calculation units; The accumulation subunit is used to accumulate the calculation result of any one of the calculation units and the calculation results of the other calculation units to obtain the output result.

[0093] Based on any of the above embodiments, the broadcast subunit is specifically used for: Based on the number of computing units in any computing cluster, evenly split the computing result of any computing unit to obtain multiple computing sub-results; Any calculation sub-result is stored in the local register of any of the calculation units, and the remaining calculation sub-results are broadcast one by one to the shared memory of other calculation units in any of the calculation clusters.

[0094] Based on any of the above embodiments, the computing task is a matrix multiplication computing task, the input data includes a left matrix and a right matrix for matrix multiplication, and the output result is an output matrix.

[0095] Based on any of the above embodiments, the first dimension among the dimensions is the row dimension of the left matrix, the second dimension among the dimensions is the column dimension of the right matrix, and the target dimension among the dimensions is the intermediate dimension, which refers to the column dimension of the left matrix or the row dimension of the right matrix.

[0096] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the computing task optimization method, which includes: determining the slice size of each dimension of the input data based on the shape of the input data of the computing task and the number of computing units on the computing hardware; dividing the computing task into multiple computing subtasks based on the slice size of each dimension, and assigning the multiple computing subtasks to each computing unit; dividing the computing units into different computing clusters based on the target dimension in each dimension and the multiple computing subtasks; calculating the computing subtask based on any computing unit in any computing cluster to obtain the computing result of the any computing unit, and accumulating the computing result of the any computing unit with the computing results of other computing units in the any computing cluster to obtain the output result, and the computing units in the any computing cluster share the computing result.

[0097] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0098] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the computing task optimization method provided by the above methods, which method includes: determining the slice size of each dimension of the input data based on the shape of the input data of the computing task and the number of computing units on the computing hardware; based on the slice size of each dimension, dividing the computing task into multiple computing subtasks, and assigning the multiple computing subtasks to each computing unit; based on the target dimension in each dimension and the multiple computing subtasks, dividing the computing units into different computing clusters; based on any computing unit in any computing cluster, calculating the computing subtask to obtain the calculation result of the any computing unit, and accumulating the calculation result of the any computing unit with the calculation results of other computing units in the any computing cluster to obtain an output result, and the computing units in any computing cluster share the calculation result.

[0099] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the computing task optimization method provided by the above-mentioned methods, the method comprising: determining the slice size of each dimension of the input data of the computing task based on the shape of the input data of the computing task and the number of computing units on the computing hardware; based on the slice size of each dimension, dividing the computing task into multiple computing subtasks, and assigning the multiple computing subtasks to each computing unit; based on the target dimension in each dimension and the multiple computing subtasks, dividing the computing units into different computing clusters; based on any computing unit in any computing cluster, calculating the computing subtask to obtain the calculation result of the any computing unit, and accumulating the calculation result of the any computing unit with the calculation results of other computing units in the any computing cluster to obtain an output result, and the computing units in the any computing cluster share the calculation result.

[0100] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0101] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A computing task optimization method, characterized in that: include: Determine the slice size of each dimension of the input data based on the shape of the input data of the computing task and the number of computing units on the computing hardware; Based on the slice size of each dimension, the computing task is divided into a plurality of computing subtasks, and the plurality of computing subtasks are allocated to each computing unit; Based on the target dimension in each dimension and the multiple computing subtasks, the computing units are divided into different computing clusters; Based on any computing unit in any computing cluster, the computing subtask is calculated to obtain a computing result of the any computing unit, and the computing result of the any computing unit is accumulated with the computing results of other computing units in the any computing cluster to obtain an output result, and the computing units in the any computing cluster share the computing results.

2. The computing task optimization method according to claim 1, characterized in that: Each dimension further includes a first dimension and a second dimension, and the computing task is divided into a plurality of computing subtasks based on the slice size of each dimension, including: Based on the slice size of the first dimension and the slice size of the second dimension, dividing the computing task into a plurality of initial subtasks; Based on the slice size of the target dimension, each initial subtask is divided to obtain the multiple computing subtasks.

3. The computing task optimization method according to claim 2, characterized in that: The dividing the computing units into different computing clusters based on the target dimension in each dimension and the plurality of computing subtasks includes: In the target dimension, computing units corresponding to all computing subtasks required to compute the same initial subtask are allocated to the same computing cluster.

4. The computing task optimization method according to claim 1, characterized in that: The computing subtask is computed based on any computing unit in any computing cluster to obtain a computing result of any computing unit, and the computing result of any computing unit is accumulated with the computing results of other computing units in any computing cluster to obtain an output result, including: In any one of the computing clusters, computing the computing subtask based on any one of the computing units, and obtaining a computing result of any one of the computing units; Broadcasting the calculation result of any one of the computing units to the shared memory of other computing units in any one of the computing clusters, so that the other computing units read the calculation result of any one of the computing units from the corresponding shared memory; Based on any one of the computing units, reading the calculation result of any one of the computing units from the local register, and reading the calculation results of each of the other computing units from the shared memory corresponding to any one of the computing units; The calculation result of any one of the calculation units is accumulated with the calculation results of the other calculation units to obtain the output result.

5. The computing task optimization method according to claim 4, characterized in that: Broadcasting the calculation result of any one of the computing units to the shared memory of other computing units in any one of the computing clusters includes: Based on the number of computing units in any computing cluster, evenly split the computing result of any computing unit to obtain multiple computing sub-results; Any calculation sub-result is stored in the local register of any of the calculation units, and the remaining calculation sub-results are broadcast one by one to the shared memory of other calculation units in any of the calculation clusters.

6. The computing task optimization method according to any one of claims 1 to 5, characterized in that: The computing task is a matrix multiplication computing task, the input data includes a left matrix and a right matrix for matrix multiplication, and the output result is an output matrix.

7. The computing task optimization method according to claim 6, characterized in that: The first dimension among the dimensions is the row dimension of the left matrix, the second dimension among the dimensions is the column dimension of the right matrix, and the target dimension among the dimensions is the intermediate dimension, which refers to the column dimension of the left matrix or the row dimension of the right matrix.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the computing task optimization method according to any one of claims 1 to 7 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computing task optimization method according to any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computing task optimization method according to any one of claims 1 to 7 is implemented.

Images