Task processing method, device and equipment

Abstract

The invention discloses a task processing method, device and equipment, and relates to the technical field of computers. The method is applied to a processor and comprises the steps that a task request is received, the task request is used for calculating the product of a first matrix and a second matrix, the first matrix comprises a plurality of first sub-blocks, and the second matrix comprises a plurality of second sub-blocks; in response to the task request, generating a plurality of calculation tasks, one calculation task being used for calculating a product of at least one sub-block pair, one sub-block pair comprising one of the first sub-blocks and one of the second sub-blocks, the product of the sub-block pair being a matrix block in a product of the first matrix and the second matrix; determining an execution sequence of the calculation tasks to enable part of sub-blocks used by adjacent calculation tasks to be the same so as to reduce cache misses; and executing each calculation task according to the execution sequence. Therefore, in the matrix multiplication calculation process, the data used among the calculation tasks are the same, that is, the data loaded in the cache region can be reused, and the hit rate can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a task processing method, apparatus, and device. Background Technology

[0002] In scenarios involving image processing in artificial intelligence models or large language models, there are numerous matrix multiplication operations. However, due to limited on-chip storage resources, handling these calculations often requires breaking down complex computational tasks and deploying them across limited hardware resources. For example, in calculating the multiplication of matrix A and matrix B to obtain matrix C, the resulting matrix C is divided into several sub-matrices. Calculating these sub-matrices requires the corresponding portions of matrix A and matrix B. This data is loaded into a cache and then moved to the core for the processing unit to execute the computational task.

[0003] However, as the size of the input matrix increases, the total read and write space required for each parallel computation will gradually exceed the size of the cache. With the reading and writing of different submatrices, the cache will be frequently swapped in and out, resulting in a decrease in the overall cache hit rate and a decrease in overall transport bandwidth, which leads to poor computational efficiency. Summary of the Invention

[0004] This application provides a task processing method, apparatus, electronic device, vehicle, computer storage medium, and computer program product that can improve cache hit rate and thus improve computing performance.

[0005] In a first aspect, embodiments of this application provide a task processing method applied to a processor of a computing device. The method includes: receiving a task request for calculating the product of a first matrix and a second matrix, the first matrix including a plurality of first sub-blocks and the second matrix including a plurality of second sub-blocks; in response to the task request, generating a plurality of computing tasks, each computing task being used to calculate the product of at least one pair of sub-blocks, each pair of sub-blocks including one of the first sub-blocks and one of the second sub-blocks, the product of the sub-block pairs being a matrix block in the product of the first matrix and the second matrix; determining the execution order of the computing tasks so that adjacent computing tasks use the same portion of the sub-blocks to reduce cache misses; and executing each computing task in the execution order.

[0006] In this way, since the data used by the computation tasks is the same during the matrix multiplication process, the data loaded in the cache can be reused, which helps to improve the hit rate.

[0007] In some implementations, in response to a task request, multiple computation tasks are generated, including obtaining a first parameter in response to the task request. The first parameter is used to characterize the row and column range threshold formed by the sub-block pairs in a single computation task in a first matrix and a second matrix. Based on the capacity of the cache and the first parameter, multiple computation tasks are generated so that the data volume of the sub-block pairs in each computation task does not exceed the capacity.

[0008] In some implementations, the processor includes multiple processor cores, and the first parameter is equal to the number of first sub-blocks or the number of second sub-blocks that can be included in a single computing task; the number of first sub-blocks and the number of second sub-blocks do not exceed the number of processor cores executing a single computing task.

[0009] In some implementations, multiple computation tasks include a first task; determining the execution order of the computation tasks to ensure that adjacent computation tasks use the same sub-blocks to reduce cache misses includes: comparing the data size of the first sub-block and the data size of the second sub-block in the first task; if the data size of the first sub-block in the first task is greater than the data size of the second sub-block, the execution order is used to ensure that the next computation task reuses the first sub-block; if the data size of the first sub-block in the first task is less than the data size of the second sub-block, the execution order is used to ensure that the next computation task reuses the second sub-block.

[0010] In some implementations, multiple computational tasks include a first task and a second task, which are one and the other of adjacent computational tasks. The computational tasks are executed in the following order: loading a first sub-block pair used by the first task into a cache, where the first sub-block pair is one or more of the sub-block pairs; executing the first task using the first sub-block pair in the cache; loading a target sub-block into the cache, where the target sub-block is a sub-block in the second task that is different from the first sub-block pair; and executing the second task using the target sub-block in the cache and the sub-blocks in the second task that are the same as those in the first sub-block pair.

[0011] In some implementations, after determining the execution order of the computation tasks, the method includes: dividing the sub-block pairs in each computation task according to the accumulation axis direction to obtain T task slices for each computation task, and using one task slice to calculate 1 / T elements in the accumulation axis direction of the sub-block pairs in its respective computation task; executing each task slice in multiple computation tasks in the execution order to obtain the corresponding slice calculation results; and accumulating the calculation results of each slice to obtain the product of the first matrix and the second matrix.

[0012] Secondly, embodiments of this application provide a task processing apparatus applied to a processor of a computing device. The apparatus includes: a receiving module for receiving a task request, the task request being used to calculate the product of a first matrix and a second matrix, the first matrix including a plurality of first sub-blocks, and the second matrix including a plurality of second sub-blocks; a processing module for generating a plurality of computing tasks in response to the task request, each computing task being used to calculate the product of at least one pair of sub-blocks, each pair of sub-blocks including one of the first sub-blocks and one of the second sub-blocks, the product of the sub-block pairs being a matrix block in the product of the first matrix and the second matrix; the processing module is further configured to determine the execution order of the computing tasks so that adjacent computing tasks use the same partial sub-blocks to reduce cache misses; and the processing module is further configured to execute each computing task according to the execution order.

[0013] In some implementations, the processing module is also used to: in response to a task request, obtain a first parameter, which is used to characterize the row and column range threshold formed by the sub-block pairs in a single computing task in the first and second matrices; and generate multiple computing tasks according to the capacity of the cache and the first parameter, so that the amount of data of the sub-block pairs in each computing task does not exceed the capacity.

[0014] In some implementations, the processor includes multiple processor cores, and the first parameter is the number of first sub-blocks or second sub-blocks that can be included in a single computing task; the number of first sub-blocks and the number of second sub-blocks do not exceed the number of processor cores executing a single computing task.

[0015] In some implementations, multiple computational tasks include a first task; the processing module is specifically used to: compare the data volume of the first sub-block and the data volume of the second sub-block in the first task; if the data volume of the first sub-block in the first task is greater than the data volume of the second sub-block, the execution order is used to make the next computational task reuse the first sub-block; if the data volume of the first sub-block in the first task is less than the data volume of the second sub-block, the execution order is used to make the next computational task reuse the second sub-block.

[0016] In some implementations, multiple computational tasks include a first task and a second task, which are one and the other of adjacent computational tasks. The processing module is specifically used to: load a first sub-block pair used by the first task into a cache, where the first sub-block pair is one or more of the sub-block pairs; execute the first task using the first sub-block pair in the cache; load a target sub-block into the cache, where the target sub-block is a sub-block in the second task that is different from the first sub-block pair; and execute the second task using the target sub-block in the cache and a portion of the sub-blocks of the first sub-block pair.

[0017] In some implementations, the processing module is also used to: divide the sub-block pairs in each computation task according to the accumulation axis direction to obtain T task slices for each computation task, and one task slice is used to calculate 1 / T elements in the accumulation axis direction of the sub-block pairs in the computation task; execute each task slice in multiple computation tasks in the order of execution to obtain the corresponding slice calculation results; and accumulate the calculation results of each slice to obtain the product of the first matrix and the second matrix.

[0018] Thirdly, embodiments of this application provide an electronic device, including: at least one memory for storing a program; at least one processor for executing the program stored in the memory; wherein, when the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0019] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0020] Fifthly, embodiments of this application provide a computer program product, characterized in that, when the computer program product is run on a processor, it causes the processor to execute the method described in the first aspect or any possible implementation of the first aspect.

[0021] In a sixth aspect, embodiments of this application provide a chip, characterized in that it includes at least one processor and an interface; the at least one processor obtains program instructions or data through the interface; the at least one processor is used to execute program line instructions to implement the method described in the first aspect or any possible implementation of the first aspect.

[0022] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0023] Figure 1A This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0024] Figure 1B This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0025] Figure 2A This is a matrix diagram illustrating the processing of multiple batches of tasks in an embodiment of this application;

[0026] Figure 2B This is a matrix diagram illustrating the processing of multiple batches of tasks in an embodiment of this application;

[0027] Figure 3A This is a matrix diagram illustrating the processing of multiple batches of tasks in an embodiment of this application;

[0028] Figure 3B This is a matrix diagram illustrating the processing of multiple batches of tasks in an embodiment of this application;

[0029] Figure 4A This is a matrix diagram illustrating the processing of task slices in an embodiment of this application;

[0030] Figure 4B This is a matrix diagram illustrating the processing of task slices in an embodiment of this application;

[0031] Figure 5 This is a flowchart illustrating a task processing method provided in an embodiment of this application;

[0032] Figure 6 This is a flowchart illustrating a task processing method provided in an embodiment of this application;

[0033] Figure 7 This is a flowchart illustrating a task processing method provided in an embodiment of this application;

[0034] Figure 8 This is a flowchart illustrating a task processing method provided in an embodiment of this application;

[0035] Figure 9 This is a flowchart illustrating a task processing method provided in an embodiment of this application;

[0036] Figure 10 This is a schematic diagram of the structure of a task processing device provided in an embodiment of this application;

[0037] Figure 11 This is a schematic diagram of the structure of a chip provided in an embodiment of this application. Detailed Implementation

[0038] In this document, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " indicates that the related objects are in an "or" relationship; for example, A / B means A or B. In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units refer to two or more processing units; multiple elements refer to two or more elements.

[0039] This corresponds to large-scale matrix multiplication calculations, such as C++. M×N =A M×K ×B K×NThe entire computation can be divided into computations of several small blocks c, each of which can be a matrix A. M×K A sub-block and matrix B K×N The product of a sub-block. Thus, the computation of each small block c is a task, and several tasks can be distributed evenly across multiple processor cores. Taking 24 cores as an example, each small block within the range of matrix C is computed by a corresponding core. In this way, the computation tasks for all small blocks can be distributed sequentially, with each core responsible for computing one small block c in turn, until matrix C is obtained. Alternatively, they can be distributed evenly, with each core responsible for multiple small blocks, until matrix C is obtained. However, since each computation requires loading all the sub-blocks needed by these cores into the cache, in scenarios involving very large-scale matrix multiplication, the cache cannot fully load all inputs, leading to frequent cache misses and impacting computational efficiency.

[0040] To improve cache hit rate and computational performance in large-scale matrix multiplication scenarios, this application provides a task processing method. This method primarily involves dividing the task based on the size of the result matrix, obtaining computation tasks for multiple sub-blocks of the result matrix, and defining the allocation method of these computation tasks so that some of the data used by consecutively executed computation tasks is the same. In this way, by reusing matrix data, the frequency of cache swapping in and out is reduced, the hit rate is improved, and thus computational performance is enhanced.

[0041] To facilitate understanding of the technical solution of this application, a computing device provided in the embodiments of this application will be described below.

[0042] For example, Figure 1A The diagram shown is a structural schematic of a computing device provided in an embodiment of this application. Figure 1A As shown, the computing device 100 may include a processor 110 and a main memory 120. The processor 110 may include a cache memory 111.

[0043] Processor 110 is the control center of computing device 100. Processor 110 can be a computing unit with computing capabilities, such as a central processing unit (CPU), graphics processing unit (GPU), data processing unit (DPU), or neural processing unit (NPU). Processor 110 includes one processing unit (also called a processing core) or multiple processing units.

[0044] The cache memory 111 is used to store instructions or data that the processing units in the processor 110 may access multiple times. This improves the speed at which the processor processes data and avoids frequent accesses of the processor 110 to the main memory 120. For example, each processing unit may have a certain cache area (which may be provided by the cache memory 111) for temporarily storing recently accessed data and instructions. This allows frequently used data to be kept closer to the processing unit (i.e., in the cache), reducing the need to read data from the slower, more power-consuming main memory and thus lowering overall energy consumption.

[0045] For example, in terms of physical form, the cache memory 111 can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), or other types of storage devices capable of storing information and instructions. In terms of logical form, the cache memory 111 can be a level 1 cache (L1 cache), a level 2 cache (L2 cache), a level 3 cache (L3 cache), or any level of cache device.

[0046] Figure 1A The connection method between the processing unit and other devices in the processor is merely illustrative. In one possible implementation, the processing unit connects to other devices in the processor via a ring bus and accesses these devices through the ring bus. In another possible implementation, the processing unit connects to other devices in the processor via a mesh bus, and the processing unit accesses these devices through the mesh bus.

[0047] Processor 110 is connected to main memory 120 via a bus. Processor 110 can perform various functions of computing device 100 by running or executing software programs stored in main memory 120 and by calling data stored in main memory 120. For example, processor 110 can execute software programs in main memory 120, call input matrices in memory 120, perform multiplication operations on the input matrices according to the strategy represented by the program, and write the operation results back to main memory 120.

[0048] The main memory 120 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, a random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or an electrically erasable programmable read-only memory (EEPROM), etc. The main memory 120 is also used to store programs related to this embodiment.

[0049] The device structure shown in Figure 1 does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0050] In some possible implementations, the computing device provided in this application embodiment can also be as follows: Figure 1B The structure shown is for reference only. Figure 1B The computing device 100 may include a host side and a device side that is communicatively connected to the host side. The host-side processor 110a may include a central processing unit (CPU) and is connected to components such as the host-side main memory 120a; the device-side processor 110b may also be referred to as an accelerator and is connected to components such as the device-side memory 120b (e.g., video memory). For example, the accelerator may include, but is not limited to, a Neural Processing Unit (NPU), a Graphics Processing Unit (GPU), and a Digital Signal Processor (DSP), and can be used to implement different types of instruction set architectures. For example, an NPU can be used as a processor to execute deep learning tasks, a GPU is mainly used for graphics rendering and computationally intensive tasks, and a DSP can be used for tasks such as digital signal processing and audio processing. The matrix multiplication task processing operation described in this embodiment can be executed by the device-side processor, but is not limited to this.

[0051] Specifically, after obtaining the two matrices A and B to be multiplied, the processor 110b on the device side can divide the two matrices A and B into their respective sub-blocks, and then combine and multiply the sub-blocks of each matrix to obtain the matrix blocks in the corresponding result matrix C. In this embodiment, the processor 110b can treat the calculation of each matrix block as a batch task, and then adopt an appropriate task allocation method so that when consecutive batches of tasks are executed, some of the data used is the same. For example, between consecutive batches of calculation tasks, the same sub-blocks of matrix A are used, but different sub-blocks of matrix B are used. The same sub-blocks can be continuously stored in the cache, and each calculation task only reloads the different sub-blocks of matrix B; or, between consecutive batches of calculation tasks, the same sub-blocks of matrix B are used, but different sub-blocks of matrix A are used. This helps to reduce the data loading time and bandwidth consumption during matrix multiplication calculations and improve the hit rate.

[0052] Next, with reference to the accompanying drawings, the principle of task processing by the processor in the embodiments of this application will be explained.

[0053] In this embodiment, reference Figure 2A As shown, processor 110b acquires the input matrix for matrix multiplication calculation. For ease of distinction, the two matrices being multiplied can be referred to as the first matrix and the second matrix, respectively. The first matrix has a size of M×K, and the second matrix has a size of K×N. If matrix A is the first matrix, then the element in the m-th row and k-th column of matrix A can be denoted as Am. mk Let m = (1, 2, 3, ..., M), k = (1, 2, 3, ..., K); if matrix B is the second matrix, then the element in the k-th row and n-th column of matrix B can be denoted as Bk. kn k = (1, 2, 3, ..., K), n = (1, 2, 3, ..., N). M, K and N are all integers greater than 0.

[0054] In this way, the processor 110b can determine the size of the result matrix C obtained by A×B based on the size of matrices A and B. The dimension of matrix C is M×N. Therefore, the processor can allocate the corresponding storage space to store the result matrix C to be calculated.

[0055] For example, to facilitate computation, before performing calculations on matrices A and B, the input matrices A and B can be dynamically partitioned. That is, they can be partitioned according to their size (number of rows and columns). Different sizes result in different matrix blocks. In some specific examples, matrix A can be partitioned into multiple row blocks A. i Each row block A i For a submatrix (also referred to in this paper as the first submatrix, or the first subblock), the row block A iThe size can be M0×K; and the matrix B is divided into multiple column blocks B j Each column block B j For a submatrix (also referred to in this paper as the second submatrix, or second subblock), column block B j The size is K×N0.

[0056] In this way, the multiplication of large-scale matrices A and B can be divided into the multiplication of several sub-block pairs, each sub-block pair including a sub-block A. i and a sub-block B j The results of multiplying each sub-block are accumulated and added to the corresponding position in the result matrix C, finally yielding the complete result matrix.

[0057] For example, the computation of each sub-block pair is a computation task, and a computation task can be assigned to one core of the processor 100b for processing. Then, multiple cores of the processor 100b can execute a batch of computation tasks in parallel to improve the computational efficiency of matrix multiplication.

[0058] The principle of matrix multiplication calculation tasks performed by the processor in the embodiments of this application is described in detail below with reference to the accompanying drawings.

[0059] In some possible implementations, processor 110b may pre-set the execution order of all batch computation tasks so that adjacent computation tasks executed by each core use the same sub-block, thereby reducing cache misses.

[0060] For example, such as Figure 2A By setting the task execution order, the row block A used between the first batch of tasks and the second batch of tasks computed in parallel by multiple cores can be configured. i Are they the same, or, as Figure 2B By setting the task execution order, the column block B used between the first batch of tasks and the second batch of tasks computed in parallel by multiple cores can be configured. j They are the same. Thus, during the calculation of matrix A and B multiplication, after the previous batch of tasks is completed, the cache can retain the same sub-blocks used by the next batch of tasks. Only the different sub-blocks used by the next batch of tasks need to be reloaded. By reusing data in the cache, the read / write time and bandwidth consumption of the cache are reduced, the amount and frequency of data being swapped in and out of the cache are reduced, and the cache hit rate is improved. It should be understood that... Figure 2A , 2B The example only demonstrates the data reuse effect of two batch computation tasks. In reality, the input matrices A and B can contain more blocks to execute more batch computation tasks.

[0061] In some possible implementations, the task width of each batch of tasks can be limited based on the cache capacity. This maximizes the amount of data reused between adjacent batches within the cache capacity, thereby improving the cache hit rate. The task width can be understood as the range of row heights or column widths formed by the computed sub-blocks in matrix A or matrix B within a single batch of tasks.

[0062] For example, Figure 2A In this context, the data reused between adjacent batches of computation tasks consists of 5 row blocks A. i That is, the task width is 5 line blocks A i The resulting row height range is 5M0; Figure 2B In this context, the data reused between adjacent batches of computation tasks consists of 7 column blocks B. j That is, the task width is 7 column blocks B j The resulting column width range is 7N0.

[0063] In this embodiment, the task width can be matched with the direction of task execution to facilitate greater data reuse. For example, if the sub-results obtained from the sequential execution of adjacent batches of tasks belong to the same row height range in the result matrix C, then the task width can represent the row height range formed by the calculated sub-block pairs in matrix A. If the sub-results obtained from the sequential execution of adjacent batches of tasks belong to the same column width range in the result matrix C, then the task width can represent the column width range formed by the calculated sub-block pairs in matrix B. Furthermore, by setting the task width, the amount of data reused between adjacent batches of computation tasks can be maximized.

[0064] In some possible implementations, the task load of X cores of processor 110b when executing a single batch computation task can be represented by the following Equation 1:

[0065]

[0066] In Equation 1, tdx is the identifier of each core, and X is the number of cores participating in the calculation; m tdx ,n tdx These are the data amounts of matrix A and matrix B loaded onto a single core, respectively.

[0067] For example, to reduce the cache load during single-batch task loading, the task width can be set during task allocation. This limits the range of sub-blocks used by a single-batch task within matrices A and B, or in other words, limits the elements within that range, thereby controlling the amount of data loaded into the cache during single-batch task execution. For instance, this task width can be set using a column width threshold N. col To achieve this, set a threshold for the column width. Therefore, the cache size Size for a single batch task can be adjusted as shown in Equation 2:

[0068]

[0069] Thus, the column width threshold N col The column width threshold N does not exceed the number of cores involved in the computation in processor 110b. col Having determined the row height range involved in a single batch task computation within matrix B, and given the fixed number of cores, the row height range involved in the same batch task computation within matrix A is also determined. Therefore, the amount of element data within this column width and row height range can be calculated, facilitating more reasonable control over the load of the single batch computation task. This ensures that the load does not exceed the cache size, thus avoiding cache misses due to insufficient data loading during computation.

[0070] For example, for a single core and the task load corresponding to a batch of tasks, in order to minimize cache misses, the task load corresponding to a batch of tasks can be filled as much as possible to maximize the cache load, provided that the task load is less than the cache. This can be expressed as follows:

[0071] Max Size

[0072] st.Size≤Cache

[0073] Here, st. refers to the constraint condition.

[0074] For example, when the load size of a single batch of tasks is less than Cache... e And fill up Cach as much as possible e In this case, to maximize the amount of data reused by adjacent batches of tasks, taking the task width as an example by setting a column width threshold, the task loading volume of a single batch can be expressed as:

[0075] Size multiplexing =K×N0×N col

[0076] Max Size (multiplexing)

[0077] st.Size≤Cache

[0078] Among them, Size multiplexing That is, the amount of data that is reused.

[0079] In other words, when calculating a single batch of tasks, this set of formulas allows the amount of data in matrix B loaded into the cache to be as large as possible, while ensuring that the amount of data in the current batch does not exceed the cache capacity. It should be understood that the task width can also be achieved by setting a row height threshold, the principle of which is similar to the column width threshold mentioned above, and will not be elaborated further.

[0080] Similarly, if the task width is limited to a row height of , it can be represented as

[0081]

[0082] Max Size (multiplexing)

[0083] st.Size≤Cache

[0084] In this way, when each batch of computation tasks is executed, the cache loads as much reusable data as possible. By reusing a large amount of data between adjacent batches of tasks, the frequency of swapping in and out of the cache is reduced, the hit rate is increased, and thus the computation efficiency is greatly improved.

[0085] In some possible implementations, such as Figure 3A As shown, each batch of computational tasks can be assigned to X cores for processing. Figure 3A In the diagram, the gray area within matrix C represents the computation task of 12 cores calculating a batch. Each core is responsible for calculating the multiplication of a first sub-block and a second sub-block. The results of each sub-computation are then filled into the storage space allocated for the result matrix C, until the result matrix is ​​obtained.

[0086] Specifically, each core 0 to 11 can have its own task queue. After the calculation order of the sub-block pairs, the calculation tasks corresponding to each matrix block in the batch can be assigned to X cores. The task queue of each core includes one calculation task from each batch of tasks. A calculation task can be a multiplication operation between the first sub-block and the second sub-block.

[0087] For example, refer to Figure 3B In the first batch of tasks (represented by the gray area), core 0 calculates subtask 11, subtask 21 calculates the product of the first row block and the first column block, core 1 calculates subtask 12 (the product of the first row block and the second column block), and so on. Following the calculation order, in the second batch of tasks (represented by the yellow area), core 0 calculates subtask 21 (the product of the fourth row block and the fourth column block), core 1 calculates subtask 22 (the product of the fourth row block and the fifth column block), and so on. These batches of tasks are assigned to their respective cores, for example, they can be placed in the task queues of each core, and the cores execute their assigned subtasks in batch order.

[0088] In this way, when the first batch of tasks is executed, the four row blocks of matrix A and the seven column blocks of matrix B are loaded into the L2 cache of the processor 110b. Each core retrieves the data required for its task from this shared L2 cache for computation. After the first batch of computation is completed, when the second batch of tasks is executed, the current seven column blocks in the L2 cache can be reused, requiring only the reloading of the three new row blocks. Similarly, when executing the third batch of tasks (represented by the green area), most of the currently stored column and row blocks in the L2 cache can be reused, requiring only the loading of the remaining necessary data. This significantly reduces the frequency of data swapping in and out of the cache, improving the cache hit rate.

[0089] In some possible implementations, after processor 110b generates corresponding computation tasks based on multiple sub-block pairs, a computation task includes multiple subtasks, and a subtask is used to compute the multiplication of a sub-block pair. However, the data size of a computation task exceeds the cache size, or there are cache limitations. s The risk, then, refer to Figure 4A As shown, the processor 110b can also divide its sub-block pairs along the accumulation axis according to each computation task, that is, divide a single computation task into multiple task slices K-Slices. e Each task slice contains smaller-grained sub-block pairs, and the common dimension of these smaller-grained sub-block pairs is 1 / T of the common dimension of the atomic block pairs, where T is greater than 1.

[0090] If the unit granularity of a row block is M0×K0 and the unit granularity of a column block is K0×N0, the size of a smaller granularity sub-block pair can also be M0×αK0 and αK0×N0, where α is an integer.

[0091] In this way, after dividing each computation task into slices, the computation results of each slice are accumulated in the corresponding direction to obtain the result matrix C.

[0092] For example Figure 4B In the example shown, processor 110b first determines multiple sub-block pairs 3M0×4N0, then splits each sub-block pair 3M0×4N0, reducing the common dimension K to K / 2. This is equivalent to splitting the original A×B=C into two smaller matrix multiplications, i.e., A D ×B D =C D A E ×B E =C E Furthermore, both of these small matrix multiplications involve combining the results of dimensionality reduction and multiplication of the common dimensions between sub-blocks in each sub-block. Therefore, the computational result C of the two small matrix multiplications is... D CE Accumulate and output the final result C.

[0093] Thus, in Cach e When the size cannot guarantee that a whole sub-block pair can be loaded, each batch of tasks can be further divided to obtain a smaller granular matrix multiplication scale. This reduces the amount of data loaded into the cache during each calculation by a factor of two, transforming multiple accumulations in a single task into multiple accumulations in multiple tasks, further reducing the amount of parallel loading on multiple cores, thereby reducing the probability of cache misses and improving computational performance.

[0094] Next, based on the above description and in conjunction with the accompanying drawings, a task allocation method provided by an embodiment of this application will be introduced. It is understood that this method is proposed based on the above description, and some or all of its content can be found in the above description.

[0095] For example, Figure 5 This diagram illustrates a task processing method according to an embodiment of this application. It is understood that this method can be executed by any device, equipment, platform, or cluster of devices with computing and processing capabilities. This example demonstrates execution on the processor of a computing device. Figure 5 As shown, the method may include:

[0096] S510, Receive a task request. The task request is used to calculate the product of a first matrix and a second matrix. The first matrix includes multiple first sub-blocks, and the second matrix includes multiple second sub-blocks.

[0097] In this step, for the two input matrices to be multiplied, namely the first matrix and the second matrix, a corresponding task request can be generated in the processor to instruct the processor to perform the multiplication operation on the first matrix and the second matrix. It should be understood that the two input matrices have a common dimension, that is, the number of rows in the first matrix is ​​equal to the number of columns in the second matrix.

[0098] For example, to facilitate calculation, the first and second matrices can be firstly divided into multiple first sub-blocks and multiple second sub-blocks. As a specific example, the first sub-block can be a row block of the first matrix, and the second sub-block can be a column block of the second matrix. For example, row block A... i The size can be M0×K, and the column block is B. j The size is K×N0.

[0099] It is understandable that the first and second sub-blocks in a sub-block pair have a common dimension, which makes multiplication between sub-blocks possible.

[0100] S520, in response to a task request, generate multiple computation tasks, one of which is used to compute the product of at least one pair of sub-blocks, the pair of sub-blocks including one of the first sub-blocks and one of the second sub-blocks, the product of the pair of sub-blocks being a matrix block in the product of the first matrix and the second matrix.

[0101] In this step, in response to the task request, the first and second sub-blocks obtained from the first and second matrices can be combined to obtain multiple sub-block pairs. Each sub-block pair is used to calculate one matrix block in the result matrix. The multiplication of a sub-block pair constitutes a subtask. Thus, one or more subtasks can form a large computational task. In other words, a computational task can obtain one or more matrix blocks in the result matrix.

[0102] In this way, the multiplication of the first matrix and the second matrix is ​​divided into the multiplication of several sub-block pairs. Then, the product of the multiplication of each sub-block pair can be used to obtain the product of the first matrix and the second matrix, which is the result matrix.

[0103] S530, determine the execution order of the computing tasks so that the partial sub-blocks used by adjacent computing tasks are the same, so as to reduce cache misses.

[0104] In this step, after generating multiple computation tasks, the execution order of these tasks can be determined first, ensuring that adjacent computation tasks use the same sub-blocks. By executing the tasks according to this order, the latter adjacent task can reuse some data loaded for the former in the cache, reducing the chance of cache misses through data reuse. For example, the first computation task includes two sub-block pairs: one pair consists of the first row block of matrix A and the first column block of matrix B, and the other pair consists of the first row block of matrix A and the second column block of matrix B; the second computation task includes two sub-block pairs: one pair consists of the second row block of matrix A and the first column block of matrix B, and the second pair consists of the second row block of matrix A and the second column block of matrix B. Since the first and second computation tasks both use the first and second column blocks of matrix B, loading these two column blocks into the cache can be used to compute these two adjacent tasks. By reusing data, the amount and frequency of data swapping in and out of the cache are reduced, which helps to improve the cache hit rate.

[0105] S540, each of the computational tasks is executed in the order described.

[0106] In this step, each computation task is executed sequentially according to the assigned execution order. When executing any computation task, all the data required for that task (i.e., the elements included in the corresponding sub-block pair) is loaded into the cache. The processor then retrieves this data from the cache and uses it to compute the current task, obtaining the corresponding result. Next, the next computation task is executed, at which point all the data required for that next computation task needs to be loaded into the cache. Since some of the data used by the two computation tasks is the same, such as using the same first sub-block or the same second sub-block, this part of the identical data can remain unchanged in the cache; only the required different data needs to be reloaded into the cache.

[0107] In this way, the data loaded in the cache can be reused multiple times during the entire calculation process of the result matrix C, which helps to reduce the amount and frequency of data swapping in and out of the cache, improve the cache hit rate, and thus improve the computing performance.

[0108] For example, the generated multiple computing tasks can be assigned to multiple cores of the processor for execution. When a computing task includes multiple subtasks, these cores can execute the various subtasks in a computing task in parallel. In this way, computing efficiency is greatly improved through multi-core block parallel computing.

[0109] In some possible implementations, such as Figure 6 The diagram shown is a flowchart illustrating a task allocation method provided in an embodiment of this application. Figure 6 As shown, it is similar to Figure 5 The main difference in the illustrated process is that it allows setting the task width to ensure that the data volume of a single computation task remains within the cache capacity and maximizes data reuse. Specifically, as shown... Figure 6 As shown, this method may also include:

[0110] S610 receives task requests.

[0111] In this embodiment, the execution principle of S610 is similar to that of S510 described above, and will not be repeated here.

[0112] S620, in response to the task request, generates multiple computing tasks.

[0113] In this embodiment, S620 is similar in execution principle to S520 described above. In particular, this step may include:

[0114] S621, in response to the task request, obtain a first parameter, the first parameter being used to characterize the row and column range thresholds formed by sub-block pairs in a single computation task in the first matrix and the second matrix.

[0115] In this example, in response to a task request, a preset first parameter can be obtained first. This parameter can limit the task width of a single calculation. The task width can be understood as the range of row height formed by the calculated sub-blocks in the first matrix or the range of column width formed in the second matrix in a single calculation task, or the range of row height formed by the calculated sub-blocks in the second matrix or the range of column width formed in the first matrix.

[0116] Therefore, the first parameter is used to characterize the row and column range thresholds formed by sub-block pairs in a single computation task in the first and second matrices.

[0117] For example, Figure 2A In the middle, the task width is set to 5 line blocks A based on the first parameter. i If the resulting row height is 5M0, then the data reused by adjacent batch computing tasks will be 5 row blocks A. i , or as Figure 2B In the middle, the task width is set to 7 column blocks B based on the first parameter. j If the resulting column width range is 7N0, then the data reused by adjacent computation tasks is 7 column blocks B. j .

[0118] S622, based on the capacity of the cache and the first parameter, generate the plurality of computing tasks so that the data amount of each computing task's sub-block pair does not exceed the capacity.

[0119] In this example, by limiting the row and column range threshold of the sub-block pairs in a single computation task in the first and second matrices by the first parameter, the amount of data of all elements in the corresponding range of the sub-block pairs in the computation task can be limited so that it does not exceed the capacity of the cache area, thus avoiding cache misses due to the cache area being unable to fully load the amount of data of the single task.

[0120] In this embodiment, the task width can be matched with the direction of task execution to facilitate the reuse of more data. For example, if the sub-results obtained by sequentially executing adjacent computation tasks belong to the same row height range in the result matrix C, then the task width can represent the row height range formed by the computed sub-block pairs in matrix A. If the sub-results obtained by sequentially executing adjacent computation tasks belong to the same column width range in the result matrix C, then the task width can represent the column width range formed by the computed sub-block pairs in matrix B. Furthermore, by setting the task width, the amount of data reused between adjacent computation tasks can be maximized, thereby improving the hit rate.

[0121] In some possible implementations, since the number of processor cores in the processor is fixed, the first parameter can be expressed as the number of first sub-blocks or second sub-blocks that can be included in a single computing task. This ensures that the number of first sub-blocks and second sub-blocks does not exceed the number of processor cores executing the single computing task. Simultaneously, the total data volume represented by the first parameter (the number of first sub-blocks or second sub-blocks) can be as large as possible. In this way, the first parameter allows for more reasonable control of the load on the computing task and helps improve the cache hit rate.

[0122] For example, the calculation methods for task width and data volume of a single task can also refer to the calculations in Equations 1 and 2 above, which will not be repeated here.

[0123] S630, determine the execution order of the computing tasks so that the partial sub-blocks used by adjacent computing tasks are the same, so as to reduce cache misses.

[0124] S640, each of the computational tasks is executed in the order described.

[0125] In this embodiment, the steps of S630 to S640 are similar to the execution principle of S530 to S540 described above, and will not be repeated here.

[0126] In some possible implementations, the computation order can be determined by identifying which two computational tasks can achieve greater data reuse. Specifically, for example... Figure 7 As shown, the method may include:

[0127] S710, compare the data volume of the first sub-block and the data volume of the second sub-block in the first task.

[0128] In this step, the first task can be any one of multiple computational tasks. Then, the total data volume of the first sub-block and the total data volume of the second sub-block in all sub-block pairs within the first task can be compared.

[0129] For example, if the first task contains 3 first sub-blocks and 4 second sub-blocks, then the data volume of the 3 first sub-blocks is compared with the data volume of the 4 second sub-blocks.

[0130] S720, if the data volume of the first sub-block in the first task is greater than the data volume of the second sub-block, then the execution order is used to make the next computing task reuse the first sub-block.

[0131] In this step, if the data volume of the first sub-block in the first task is greater than the data volume of the second sub-block, the allocated execution order is used to ensure that the next computation task of the first task reuses the first sub-block.

[0132] S730, if the data volume of the first sub-block in the first task is less than the data volume of the second sub-block, then the execution order is used to make the next computing task reuse the second sub-block.

[0133] In this step, similar to the principle of step S720, if the data volume of the first sub-block in the first task is greater than that of the second sub-block, it means that reusing the second sub-block in the next calculation is more conducive to reducing the probability of cache miss. Therefore, the execution order can be made so that the next calculation task reuses the second sub-block, thereby reducing cache consumption and improving cache hit rate.

[0134] In some possible implementations, the computational task generated by the sub-block pair may include a first task and a second task, where the first task and the second task are one and the other of adjacent tasks. Therefore, this method can be specifically implemented as follows: Figure 8 The process shown is as follows: Figure 8 As shown, the method may include:

[0135] S801 receives task requests.

[0136] S802, in response to the task request, generates multiple computing tasks.

[0137] S803, determine the execution order of the computing tasks so that the partial sub-blocks used by adjacent computing tasks are the same.

[0138] In this embodiment, the execution principles of S801 to S803 are similar to those of S510 to S530 described above, and will not be repeated here.

[0139] S804, Execute each of the computational tasks according to the execution order. Specifically, this step may include:

[0140] S8041, load the first sub-block pair used by the first task into the cache area, wherein the first sub-block pair is one or more of the sub-block pairs.

[0141] In this step, taking any two adjacent computation tasks, namely the first task and the second task, as an example, the first task is executed first, followed by the second task, according to the execution order. When the first task is executed, all element data of the first sub-block pair required by the first task are loaded into the buffer. The first sub-block pair is one or more of all combined sub-block pairs.

[0142] S8042, using the first sub-block pair of the cache area, execute the first task.

[0143] In this step, after the processor retrieves the data of the first sub-block pair from the cache, it executes the first task, which performs the multiplication operation on each of the first sub-block pairs.

[0144] S8043, load the target sub-block into the cache area, wherein the target sub-block is a sub-block in the second task that is different from the first sub-block pair.

[0145] In this step, after completing the first task, the processor continues to execute the second task. At this time, the sub-block pairs required by the second task need to be loaded into the cache for the processor to retrieve. However, since the two tasks have the same sub-block pairs, only the different sub-blocks, i.e., the target sub-block pairs, can be reloaded into the cache. Then, the identical sub-block pairs in the cache can be directly reused, reducing the amount of data swapped in and out of the cache and shortening the loading time.

[0146] S8044, the second task is executed using the target sub-block in the cache area and the sub-block in the second task that is the same as the first sub-block pair.

[0147] In this step, the processor searches for the target sub-block in the cache, as well as the sub-block in the second task that is identical to the first sub-block, executes the second task, and obtains the corresponding product. This process continues until all computational tasks are completed.

[0148] In some possible implementations, the sub-blocks can be further divided along the accumulation axis of the matrix to perform matrix multiplication calculations at a finer granularity. Specifically, for example... Figure 9 As shown, the method may specifically include:

[0149] S901 receives task requests.

[0150] S902, in response to the task request, generate multiple computing tasks.

[0151] S903, determine the execution order of the computing tasks so that the partial sub-blocks used by adjacent computing tasks are the same.

[0152] In this embodiment, the execution principles of S901 to S903 are similar to those of S510 to S530 described above, and will not be repeated here.

[0153] S904, the sub-block pairs in each computing task are divided according to the accumulation axis direction to obtain N task slices for each computing task. Each task slice is used to calculate the 1 / N elements of the sub-block pairs in the accumulation axis direction of the computing task.

[0154] In this step, if the amount of data in the computation task still exceeds the cache size after the input matrix is ​​partitioned, the processor divides the sub-block pairs in each computation task along the accumulation axis direction to obtain N task slices for each computation task. One task slice is used to calculate 1 / N elements of the sub-block pairs in its respective computation task along the accumulation axis direction.

[0155] In this way, the number of rows and columns of the resulting matrix blocks remains unchanged compared to before the split, but the common dimension of the first and second sub-blocks in the sub-block pair is reduced by equal division. For example, in the example shown in the figure, the task of multiplying matrix blocks A1 and B1 is split into multiplying matrix blocks A11 and B11, and multiplying matrix blocks A12 and B12.

[0156] S905, according to the execution order, each task slice in the plurality of computing tasks is executed sequentially to obtain the corresponding slice calculation results.

[0157] In this step, after splitting the tasks in each batch, each batch forms the same number of subtask sets, and the computation result of each subtask set is a matrix block in matrix C. Then, assigning the subtasks of each batch to the corresponding cores for execution yields several sub-result matrices.

[0158] S906, sum up the calculation results of each slice to obtain the product of the first matrix and the second matrix.

[0159] In this step, several sub-result matrices are accumulated to obtain the final matrix C. Thus, when the cache size cannot guarantee the full loading of a single sub-block pair, each batch of tasks can be further divided to obtain smaller-granularity matrix multiplication scales. This significantly reduces the amount of data loaded into the cache during each calculation, transforming single-task multiple accumulations into multi-task multiple accumulations, further reducing the amount of parallel loading across multiple cores, thereby lowering the probability of cache misses and improving computational performance.

[0160] Based on the methods described in the above embodiments, this application provides an apparatus. Please refer to... Figure 10 , Figure 10 This is a schematic diagram of the structure of a device 1100 provided in an embodiment of this application.

[0161] like Figure 10The device 1100 is applied to a processor and may specifically include: a receiving module 1101 for receiving a task request, the task request being used to calculate the product of a first matrix and a second matrix, the first matrix including a plurality of first sub-blocks and the second matrix including a plurality of second sub-blocks; a processing module 1102 for generating a plurality of computing tasks in response to the task request, one of the computing tasks being used to calculate the product of at least one pair of sub-blocks, one pair of sub-blocks including one of the first sub-blocks and one of the second sub-blocks, the product of the pair of sub-blocks being a matrix block in the product of the first matrix and the second matrix; the processing module 1102 is further used to determine the execution order of the computing tasks so that adjacent computing tasks use the same portion of the sub-blocks to reduce cache misses; the processing module 1102 is further used to execute each computing task according to the execution order.

[0162] In some implementations, the processing module 1102 is further configured to: in response to the task request, obtain a first parameter, the first parameter being used to characterize the row and column range thresholds formed by sub-block pairs in a single computation task in the first matrix and the second matrix;

[0163] Based on the capacity of the cache and the first parameter, the plurality of computing tasks are generated so that the amount of data in the sub-block pairs of each computing task does not exceed the capacity.

[0164] In some implementations, the plurality of computing tasks includes a first task. The processing module 1102 is specifically used to: compare the data volume of the first sub-block and the data volume of the second sub-block in the first task; if the data volume of the first sub-block in the first task is greater than the data volume of the second sub-block, the execution order is used to make the next computing task reuse the first sub-block; if the data volume of the first sub-block in the first task is less than the data volume of the second sub-block, the execution order is used to make the next computing task reuse the second sub-block.

[0165] In some implementations, the plurality of computing tasks includes a first task and a second task, wherein the first task and the second task are one and the other of the adjacent computing tasks;

[0166] The processing module is specifically used for:

[0167] Load the first sub-block pair used by the first task into the cache area, wherein the first sub-block pair is one or more of the sub-block pairs;

[0168] The first task is executed using the first sub-block pair in the cache area;

[0169] The target sub-block is loaded into the cache area, and the target sub-block is a sub-block in the second task that is different from the first sub-block pair;

[0170] The second task is executed using the target sub-block of the cache and a portion of the sub-blocks of the first sub-block pair.

[0171] For example, the processing module is further configured to:

[0172] The sub-block pairs in each computation task are divided along the accumulation axis direction to obtain N task slices for each computation task. Each task slice is used to calculate 1 / N elements of the sub-block pairs in the accumulation axis direction of the computation task.

[0173] According to the execution order, each task slice in the plurality of computing tasks is executed sequentially to obtain the corresponding slice computing results;

[0174] The calculation results of each slice are summed to obtain the product of the first matrix and the second matrix.

[0175] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0176] Based on the methods in the above embodiments, this application provides an electronic device. The electronic device may include: at least one memory for storing a program; and at least one processor for executing the program stored in the memory; wherein, when the program stored in the memory is executed, the processor executes the methods in the above embodiments.

[0177] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0178] Based on the methods in the above embodiments, this application provides a computer program product, characterized in that, when the computer program product is run on a processor, the processor executes the methods in the above embodiments.

[0179] Based on the methods described in the above embodiments, this application also provides a chip. Please refer to... Figure 11 , Figure 11 This is a schematic diagram of a chip structure provided in an embodiment of this application. Figure 11 As shown, chip 900 includes one or more processors 901 and interface circuitry 902. Optionally, chip 900 may also include a bus 903. Wherein:

[0180] The processor 901 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed through integrated logic circuits in the hardware of the processor 901 or through software instructions. The processor 901 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods and steps disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor.

[0181] The interface circuit 902 can be used to send or receive data, instructions or information. The processor 901 can use the data, instructions or other information received by the interface circuit 902 to process the data, instructions or other information, and can send the processed information out through the interface circuit 902.

[0182] Optionally, chip 900 also includes memory, which may include read-only memory and random access memory, and provides operation instructions and data to the processor. A portion of the memory may also include non-volatile random access memory (NVRAM).

[0183] Optionally, the memory stores executable software modules or data structures, and the processor can execute corresponding operations by calling the operation instructions stored in the memory (which may be stored in the operating system).

[0184] Optionally, the interface circuit 902 can be used to output the execution results of the processor 901.

[0185] It should be noted that the functions of the processor 901 and the interface circuit 902 can be implemented through hardware design, software design, or a combination of hardware and software; no restrictions are imposed here.

[0186] It should be understood that each step of the above method embodiments can be completed by hardware logic circuits or software instructions in a processor.

[0187] It is understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. In addition, in some possible implementations, each step in the above embodiments may be selectively executed according to the actual situation, and may be partially or fully executed, which is not limited here.

[0188] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0189] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0190] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0191] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

[0192] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A task processing method, characterized in that, The method is applied to a processor of a computing device, and the method includes: Receive a task request, the task request being used to calculate the product of a first matrix and a second matrix, the first matrix comprising a plurality of first sub-blocks, and the second matrix comprising a plurality of second sub-blocks; In response to the task request, a plurality of computation tasks are generated, one of the computation tasks being used to compute the product of at least one pair of sub-blocks, one pair of sub-blocks including one of the first sub-blocks and one of the second sub-blocks, the product of the pair of sub-blocks being a matrix block in the product of the first matrix and the second matrix; The execution order of the computation tasks is determined so that adjacent computation tasks use the same sub-blocks, thereby reducing cache misses; Each computational task is executed in the order described.

2. The method according to claim 1, characterized in that, In response to the task request, multiple computing tasks are generated, including: In response to the task request, a first parameter is obtained, which is used to characterize the row and column range threshold formed by the sub-block pairs in the first matrix and the second matrix in a single computing task. Based on the capacity of the cache and the first parameter, the plurality of computing tasks are generated so that the amount of data in the sub-block pairs of each computing task does not exceed the capacity.

3. The method according to claim 1 or 2, characterized in that, The processor includes multiple processor cores. The first parameter is equal to the number of first sub-blocks or the number of second sub-blocks that can be included in a single computing task; The number of the first sub-block and the number of the second sub-block do not exceed the number of processor cores executing a single computing task.

4. The method according to any one of claims 1-3, characterized in that, The plurality of computing tasks includes a first task; Determining the execution order of the computation tasks to ensure that adjacent computation tasks use the same sub-blocks, thereby reducing cache misses, includes: Compare the data volume of the first sub-block and the data volume of the second sub-block in the first task; If the data volume of the first sub-block in the first task is greater than the data volume of the second sub-block, then the execution order is used to make the next computing task reuse the first sub-block; If the amount of data in the first sub-block of the first task is less than the amount of data in the second sub-block, then the execution order is used to make the next computing task reuse the second sub-block.

5. The method according to any one of claims 1-4, characterized in that, The plurality of computing tasks includes a first task and a second task, wherein the first task and the second task are one and the other of the adjacent computing tasks; The execution of each computational task in the specified execution order includes: Load the first sub-block pair used by the first task into the cache area, wherein the first sub-block pair is one or more of the sub-block pairs; The first task is executed using the first sub-block pair in the cache area; The target sub-block is loaded into the cache area, and the target sub-block is a sub-block in the second task that is different from the first sub-block pair; The second task is executed using the target sub-block in the cache area and the sub-block in the second task that is the same as the first sub-block pair.

6. The method according to any one of claims 1-5, characterized in that, After determining the execution order of the computational tasks, the method includes: The sub-block pairs in each computation task are divided along the accumulation axis direction to obtain T task slices for each computation task. Each task slice is used to calculate the 1 / T element in the accumulation axis direction of the sub-block pairs in its respective computation task. According to the execution order, each task slice in the plurality of computing tasks is executed sequentially to obtain the corresponding slice computing results; The calculation results of each slice are summed to obtain the product of the first matrix and the second matrix.

7. A task processing device, characterized in that, The device is applied to a processor of a computing device, and the device includes: A receiving module is used to receive a task request, the task request being used to calculate the product of a first matrix and a second matrix, the first matrix comprising a plurality of first sub-blocks and the second matrix comprising a plurality of second sub-blocks; A processing module is configured to generate multiple computation tasks in response to the task request, one of the computation tasks being configured to compute the product of at least one pair of sub-blocks, the pair of sub-blocks including one of the first sub-blocks and one of the second sub-blocks, the product of the pair of sub-blocks being a matrix block in the product of the first matrix and the second matrix; The processing module is also used to determine the execution order of the computing tasks so that adjacent computing tasks use the same sub-blocks, thereby reducing cache misses; The processing module is also used to execute each of the computational tasks in the execution order.

8. The task processing apparatus according to claim 7, characterized in that, The processing module is also used for: In response to the task request, a first parameter is obtained, which is used to characterize the row and column range threshold formed by the sub-block pairs in the first matrix and the second matrix in a single computing task. Based on the capacity of the cache and the first parameter, the plurality of computing tasks are generated so that the amount of data in the sub-block pairs of each computing task does not exceed the capacity.

9. The task processing apparatus according to claim 7 or 8, characterized in that, The processor includes multiple processor cores. The first parameter is the number of first sub-blocks or the number of second sub-blocks that can be included in a single computing task; The number of the first sub-block and the number of the second sub-block do not exceed the number of processor cores executing a single computing task.

10. The apparatus according to any one of claims 7-9, characterized in that, The plurality of computing tasks includes a first task; The processing module is specifically used for: Compare the data volume of the first sub-block and the data volume of the second sub-block in the first task; If the data volume of the first sub-block in the first task is greater than the data volume of the second sub-block, then the execution order is used to make the next computing task reuse the first sub-block; If the amount of data in the first sub-block of the first task is less than the amount of data in the second sub-block, then the execution order is used to make the next computing task reuse the second sub-block.

11. The method according to any one of claims 7-10, characterized in that, The plurality of computing tasks includes a first task and a second task, wherein the first task and the second task are one and the other of the adjacent computing tasks; The processing module is specifically used for: Load the first sub-block pair used by the first task into the cache area, wherein the first sub-block pair is one or more of the sub-block pairs; The first task is executed using the first sub-block pair in the cache area; The target sub-block is loaded into the cache area, and the target sub-block is a sub-block in the second task that is different from the first sub-block pair; The second task is executed using the target sub-block of the cache and a portion of the sub-blocks of the first sub-block pair.

12. The method according to any one of claims 7-11, characterized in that, The processing module is also used for: The sub-block pairs in each computation task are divided along the accumulation axis direction to obtain T task slices for each computation task. Each task slice is used to calculate the 1 / T element in the accumulation axis direction of the sub-block pairs in its respective computation task. According to the execution order, each task slice in the plurality of computing tasks is executed sequentially to obtain the corresponding slice computing results; The calculation results of each slice are summed to obtain the product of the first matrix and the second matrix.

13. A computing device, characterized in that, Each computing device includes a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1-6.

14. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster causes the computing device cluster to perform the method as described in any one of claims 1-6.

Images