Matrix multiplication task execution method, apparatus, device, medium, and program product

Abstract

This application discloses a matrix multiplication task execution method, apparatus, device, medium, and program product. The method is applied to a processor unit in a graphics processing unit. The method includes: during the execution of a matrix multiplication task in the processor unit, allocating a target register for the matrix multiplication task in the processor unit, the target register including a shared register; generating at least two types of second thread bundles for executing the matrix multiplication task; the at least two types of second thread bundles asynchronously executing the matrix multiplication task based on the shared register; and the shared register being used to store intermediate results of the matrix multiplication task.

Description

Technical Field

[0001] This application relates to the field of processor technology, and in particular to a matrix multiplication task execution method, apparatus, device, storage medium, and computer program product. Background Technology

[0002] With the rapid development of artificial intelligence and high-performance computing, matrix multiplication, as a core operation in computationally intensive tasks, directly impacts the performance of applications such as deep learning training and scientific simulations. Graphics processors (GPUs), with their massively parallel computing architecture, have become a key hardware platform for accelerating matrix multiplication. The high-dimensional data interactions and computational dependencies involved in matrix multiplication pose significant challenges to GPU hardware resource management and thread scheduling mechanisms. Summary of the Invention

[0003] In view of the above, embodiments of this application provide at least one matrix multiplication task execution method, apparatus, device, storage medium, and program product.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] On one hand, embodiments of this application provide a matrix multiplication task execution method, applied to a processor unit in a graphics processor. The method includes: during the execution of a matrix multiplication task by the processor unit, allocating a target register for the matrix multiplication task in the processor unit, the target register including a shared register; generating at least two types of second thread bundles for executing the matrix multiplication task; the at least two types of second thread bundles asynchronously executing the matrix multiplication task based on the shared register; the shared register being used to store intermediate results of the matrix multiplication task.

[0006] On the other hand, embodiments of this application provide a matrix multiplication task execution apparatus, applied to a processor unit in a graphics processor. The apparatus includes: an execution module, configured to allocate a target register for the matrix multiplication task in the processor unit during the execution of the matrix multiplication task in the processor unit, the target register including a shared register; and a generation module, configured to generate at least two types of second thread bundles for executing the matrix multiplication task; the at least two types of second thread bundles asynchronously execute the matrix multiplication task based on the shared register; and the shared register is used to store intermediate results of the matrix multiplication task.

[0007] In another aspect, embodiments of this application provide a computer device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the program to implement some or all of the steps in the above-described method.

[0008] In another aspect, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements some or all of the steps in the above-described method.

[0009] In another aspect, embodiments of this application provide a computer program product, including a computer program or instructions, which, when executed by a processor, implement some or all of the steps in the above-described method.

[0010] In this embodiment, when the processor unit executes a matrix multiplication task, by allocating a target register for the matrix multiplication task and generating at least two types of second thread bundles, effective management of task resources and parallel processing of the task can be achieved. Simultaneously, by enabling at least two types of second thread bundles to execute the matrix multiplication task asynchronously, and utilizing the target register, which includes a shared register, to store intermediate results during execution, thread bundles with different functions can work collaboratively. Furthermore, the shared register is visible to all types of thread bundles, avoiding frequent movement of intermediate results between the on-chip shared buffer and the thread bundle's private registers, thereby saving on-chip bandwidth. Based on the embodiments provided in this application, the execution efficiency of the matrix multiplication task can be significantly improved, resource overhead reduced, and the overall processing capability of the processor unit enhanced through asynchronous parallel execution of tasks.

[0011] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this application. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0013] Figure 1A A schematic diagram of the architecture of a graphics processor provided in an embodiment of this application;

[0014] Figure 1B A schematic diagram of the architecture of a processor unit provided for an embodiment of this application;

[0015] Figure 1C A schematic diagram of matrix multiplication provided in an embodiment of this application;

[0016] Figure 1D A schematic diagram illustrating data loading delay during matrix multiplication, provided as an embodiment of this application;

[0017] Figure 1E A schematic diagram of a matrix multiplication process provided in this application embodiment;

[0018] Figure 2This application provides a schematic diagram of an optimized matrix multiplication pipeline.

[0019] Figure 3 A schematic diagram of another optimized matrix multiplication provided in this application embodiment;

[0020] Figure 4 A schematic diagram illustrating the implementation flow of a matrix multiplication task execution method provided in this application embodiment;

[0021] Figure 5 A schematic diagram of the implementation process of a matrix multiplication task execution method provided in this application embodiment. Figure 2 ;

[0022] Figure 6 A schematic diagram of the execution flow of a thread bundle provided in an embodiment of this application;

[0023] Figure 7 A register allocator architecture provided for embodiments of this application;

[0024] Figure 8 A schematic diagram of an asynchronous thread waiting scheme based on arrival instructions and waiting instructions provided in an embodiment of this application;

[0025] Figure 9 This is a schematic diagram of the composition structure of a matrix multiplication task execution device provided in an embodiment of this application;

[0026] Figure 10 This is a schematic diagram of the hardware entity of a computer device provided in an embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] In the following description, references to "some embodiments" refer to a subset of all possible embodiments. It is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict. The terms "first / second / third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this application.

[0030] (1) Graphics Processing Unit (GPU): A processor specifically designed for handling graphics and computational tasks. Unlike traditional CPUs, GPUs have a large number of parallel computing units (such as CUDA cores or AMD's CU), enabling them to execute multiple computational tasks simultaneously. The design of GPUs makes them well-suited for handling large-scale data-parallel computing tasks, such as graphics rendering, machine learning, and scientific computing.

[0031] (2) Kernel: Program code executed on the GPU, usually a function or a piece of code. The Kernel is the core of parallel computing, defining the operations to be performed. When a Kernel is invoked, it generates a large number of Work-items to perform specific computational tasks. The execution of the Kernel is organized and scheduled through Work-groups.

[0032] (3) Work-group: A logical unit on the GPU, consisting of multiple work-items. The main function of a work-group is to divide the kernel's execution tasks into smaller units in order to better utilize the GPU's parallel computing capabilities.

[0033] (4) Wave: Wave is the basic unit of thread scheduling on the GPU, usually consisting of 32 or 64 work-items (depending on the GPU architecture). Wave is a hardware-level execution unit, and multiple Waves can be executed simultaneously on different computing units.

[0034] (5) Thread (Work-item): The smallest unit of execution of a task on the GPU, usually corresponding to a single thread or an element in SIMD. Each Work-item has a unique ID and executes the computational tasks defined in the Kernel.

[0035] The matrix multiplication task provided in this application is implemented using a graphics processor. In some embodiments, refer to... Figure 1ATaking a graphics processor as an example. The graphics processor 100 includes multiple processor clusters (PCs) 101 (e.g., 4), each processor cluster 101 may include multiple processor execution engines (PXs) 102 (e.g., 4), and each processor execution engine 102 may include multiple processor units (Ps) 103 (e.g., 2).

[0036] Multiple processor execution engines 102 in each processor cluster 101 share an L2 cache 104. The main memory of the graphics processor 100 is implemented as an L3 cache 105.

[0037] refer to Figure 1B Each processor unit 103 may include: a tensor calculation engine 1031, a tensor storage engine 1032, an L1 cache 1033, and a register 1034.

[0038] Please see Figure 1C This diagram illustrates a matrix multiplication operation. Let matrix A have dimensions M*K and matrix B have dimensions N*K. In parallel acceleration architectures such as GPUs, matrix multiplication is typically broken down into smaller chunks. Each C_tile is placed in a Processor (P0) to calculate the result. The calculation process requires accumulating C_tile0 = A0*B0 + A1*B1 + A2*B2 + ... + A n-1 *B n-1 .

[0039] During the accumulation process, each loop (K loop) typically loads A. K (load A) K ) -> Load B K (loadB) K )->Accumulate (A) K *B K (calculate A) K *B K +ACC). In the original implementation, the data loading process would be exposed, such as Figure 1D Data loading delay in the process.

[0040] In related technologies, optimization strategies for GPU matrix multiplication typically involve multi-level buffer prefetching and using soft pipelining to hide latency, such as... Figure 1E An optimization scheme using a double buffer as an example.

[0041] Ideally, in a double buffer programming model, the same thread wave issues a data loading instruction at the beginning of the Kth loop, loading the data from the K+1th loop, and then waits for the data sent in the previous loop to be ready before starting calculation and accumulation (A). K *B K (calculate A) K *B K +ACC). After the calculation is complete, the next loop begins, issuing an instruction to load the data used in the K+2th loop, and so on.

[0042] However, the software pipelining schemes with multi-level buffers in related technologies have at least the following problems:

[0043] (1) Although the data for the next cycle is preloaded into the on-chip buffer via pipelined loading, there is a clear synchronization relationship between the LOAD instruction and the computation instruction in a hot loop. The loading of the next data needs to wait for the current computation instruction to complete, such as... Figure 2 The waiting interval time in the process. Also, because the same wave needs to issue load and compute instructions, the issuance of load instructions may also affect the start time of compute instructions.

[0044] (2) At the end of the hot loop, the accumulated sum is usually post-processed and then written out. The same processor will start the execution of subsequent tiles, repeating the pre-loading (prolog) -> hot loop -> post-processing (epilog) process for each tile. The time for post-processing (epilog) and pre-loading (prolog) will be exposed between two tiles, such as... Figure 2 As processor computing density increases, the interval between two tiles in the process becomes faster, and the exposure time at both ends becomes more severe. This leads to a significant decrease in the overall computing efficiency of the processor.

[0045] To address the aforementioned technical problems, this application provides a novel programming model that allows multiple threads (waves) within the same kernel to asynchronously execute different data loading, computation, post-processing, and writing processes. This eliminates time waste caused by false dependencies between stages and maximizes the overlap of time between stages to achieve the highest computational efficiency.

[0046] In some embodiments, multiple waves executing the same kernel (code segment) have the same configuration (mainly register file size), and the private registers of multiple waves are isolated from each other, with only the shared on-chip buffer visible. However, in the asynchronous programming model, the Load wave only needs to load data, calculate the address, and issue a LOAD instruction; the Math wave needs to perform calculations and store the multiplication-accumulation result in the register file; and the Store wave processes the multiplication-accumulation result from the register file and writes it out.

[0047] like Figure 3 As shown, after the loading task completes loading A0 and B0, it indicates that the data for the 0th loop is ready, and the calculation task can perform the accumulation process for the 0th loop (ACC += A0 * B0). Similarly, after the loading task completes loading A1 and B1, it indicates that the data for the 1st loop is ready, and the calculation task can perform the accumulation process for the 1st loop (ACC += A1 * B1), and so on. After the calculation task completes the accumulation process for the Kth loop, the hot loop of tile0 ends, and the storage task performs the subsequent post-processing (epilog), as shown... Figure 3 In post-processing and quantization, D = ACC * scaling + offset.

[0048] If the multiplication and accumulation result is stored in the on-chip shared buffer, it is inevitable that the computation task (Mathwave) and storage task (Storewave) will need to move the multiplication and accumulation result back and forth between the on-chip shared buffer and the register file, wasting bandwidth.

[0049] Considering that the three different types of wave execute different business operations, the required register resources are also different. If the register resources are allocated according to the largest register requirement among the three business operations, it will be a waste of register resources.

[0050] This application provides a matrix multiplication task execution method, which can be executed by the processor of a computer device. The computer device refers to a device with data processing capabilities, such as a server, laptop, tablet, desktop computer, smart TV, set-top box, or mobile device (e.g., mobile phone, portable video player, personal digital assistant, dedicated messaging device, portable gaming device).

[0051] Figure 4 This is a schematic diagram illustrating the implementation process of a matrix multiplication task execution method provided in an embodiment of this application, as shown in Figure 1. Figure 4 As shown, the method includes the following steps S401 and S402, combining... Figure 4The steps shown are explained.

[0052] Step S401: During the execution of the matrix multiplication task in the processor unit, a target register is allocated for the matrix multiplication task in the processor unit, and the target register includes a shared register.

[0053] Step S402: Generate at least two types of second thread bundles for performing the matrix multiplication task; the at least two types of second thread bundles asynchronously execute the matrix multiplication task based on the shared register; the shared register is used to store the intermediate results of the matrix multiplication task.

[0054] Here, the at least two types of second thread bundles execute the matrix multiplication task asynchronously; the target register includes a shared register that is visible to the at least two types of second thread bundles, and the shared register is used to store intermediate results in the execution of the matrix multiplication task; in some embodiments, the intermediate result is a result obtained during the execution of the matrix multiplication task but not as the final output, for example, the intermediate result can be a multiplication-accumulation result.

[0055] In some embodiments, please refer to Figure 1B The processor unit 103 may be equipped with a register 1034. During the execution of the matrix multiplication task by the processor unit, the corresponding target register can be allocated from the register 1034 for the matrix multiplication task.

[0056] The matrix multiplication task involves performing multiplication operations on two input matrices (e.g., the first and second matrices in subsequent embodiments, exemplarily having dimensions of M×K and N×K, respectively) on a graphics processor to obtain a target matrix. In some embodiments, the multiplication task of the two input matrices can be split into multiple subtasks based on the target matrix, each subtask processing a block of matrix data within the target matrix. It is understood that the matrix multiplication task here can be either a multiplication task to obtain the target matrix or a multiplication task to obtain a block of matrix data within the target matrix.

[0057] In some embodiments, the number of thread bundles in a type of second thread bundle can be one or more, and the number of second thread bundles in different types can be the same or different. For example, if there are second thread bundles of type A and second thread bundles of type B, the number of second thread bundles of type A can be one, and the number of thread bundles of type B can be five. It is understood that a thread bundle can include multiple threads.

[0058] In some embodiments, the above-described at least two types of second thread bundles are used to asynchronously execute the matrix multiplication task. Within one type of second thread bundle, the individual thread bundles can perform the same function, while different types of second thread bundles can perform different functions.

[0059] To understand the relationship between the various second thread bundles, the following explanation uses the example of three types of second thread bundles:

[0060] For example, the above-mentioned at least two types of second thread bundles may include a loading thread bundle, a computation thread bundle, and a storage thread bundle. The loading thread bundle is used to perform data preloading (e.g., loading a matrix block from system memory into shared memory); the computation thread bundle is used to perform multiply-accumulate (MAC) operations; and the storage thread bundle is used to write the result back to system memory.

[0061] In the current scenario, the matrix multiplication task described above is decoupled into three stages: loading, computation, and storage. Parallel execution of the matrix multiplication task is achieved by executing these three stages in parallel. For example, please refer to... Figure 3 The following processes can be executed in parallel: loading A1B1 of the loading thread bundle processing tile1, multiplying and accumulating A0B0 of the loading thread bundle processing tile1, and storing the post-processing and storage of the loading thread bundle processing tile0.

[0062] Considering the need to move intermediate results (such as multiplication-accumulation results) during matrix multiplication calculations back and forth between the on-chip shared buffer and the private registers of each thread bundle when registers are isolated between thread bundles, in order to save on-chip bandwidth, in this embodiment of the application, a target register can be allocated for each second thread bundle in the processor unit's registers, and this target register includes a shared register. By storing the aforementioned intermediate results in this shared register, since the shared register is visible to the at least two types of second thread bundles, there is no need to move intermediate results during matrix multiplication calculations back and forth between the on-chip shared buffer and the private registers of each thread bundle, thus saving on-chip bandwidth.

[0063] In this embodiment, when the processor unit executes a matrix multiplication task, by allocating a target register for the matrix multiplication task and generating at least two types of second thread bundles, effective management of task resources and parallel processing of the task can be achieved. Simultaneously, by enabling at least two types of second thread bundles to execute the matrix multiplication task asynchronously, and utilizing the target register, which includes a shared register, to store intermediate results during execution, thread bundles with different functions can work collaboratively. Furthermore, the shared register is visible to all types of thread bundles, avoiding frequent movement of intermediate results between the on-chip shared buffer and the thread bundle's private registers, thereby saving on-chip bandwidth. Based on the embodiments provided in this application, the execution efficiency of the matrix multiplication task can be significantly improved, resource overhead reduced, and the overall processing capability of the processor unit enhanced through asynchronous parallel execution of tasks.

[0064] In some embodiments, the method further includes: generating a first thread bundle in response to the matrix multiplication task being assigned to the processor unit; the task executed by the first thread bundle prepares for executing the task of the second thread bundle.

[0065] In some embodiments, the first thread bundle is used to execute first-stage instructions; at least two types of second thread bundles are used to execute second-stage instructions; the matrix multiplication task is implemented through the first-stage instructions and the second-stage instructions. The shared register is visible to the first thread bundle; the first-stage instructions are dependent on the second-stage instructions.

[0066] In this embodiment, the instructions corresponding to the matrix multiplication task may include a first-stage instruction and a second-stage instruction. The first-stage instruction can at least complete resource initialization, and the second-stage instruction can at least complete asynchronous parallel computation. Since the execution of the second-stage instruction requires access to at least the registers initialized by the first-stage instruction, the completion of the first-stage instruction is a necessary condition for the subsequent initiation of the second-stage instruction. The task executed by the first thread bundle prepares for the execution of the task of the second thread bundle. In other words, there is a dependency between the first-stage instruction and the second-stage instruction, and the first-stage instruction is dependent on the second-stage instruction.

[0067] The first-stage instruction is also used to generate common data that each of the second thread bundles needs to use.

[0068] Considering that different types of second thread bundles need to use the same shared data during the matrix multiplication task, in order to avoid the resource consumption of generating the shared data for each type of second thread bundle, this application embodiment sets a first thread bundle and generates the shared data uniformly through the first thread bundle and stores it in a shared register. In this way, the various types of second thread bundles generated later can directly read the shared data from the shared register.

[0069] In some embodiments, when the current matrix multiplication task is used to compute a matrix block in the target matrix, the shared data includes at least the position range of the matrix block in the target matrix.

[0070] Understandably, this shared register is visible to both the first and second thread bundles.

[0071] In some embodiments, when the processor unit processes two matrix multiplication tasks simultaneously, the target registers corresponding to the different matrix multiplication tasks are independent / isolated from each other.

[0072] In some embodiments, the target register is allocated by the first thread bundle in the processor unit for the matrix multiplication task, and the shared register is visible to the first thread bundle.

[0073] Here, the matrix multiplication task is assigned to the processor unit. The processor unit responds to the task assignment by generating the first thread bundle and allocating a target register for the matrix multiplication task. The target register is visible to the first thread bundle (at this time, there is no second thread bundle corresponding to the matrix multiplication task).

[0074] In some embodiments, during the allocation of the target register, the target register is further divided, at least into a common register and a private register corresponding to the first thread bundle. The first thread bundle can access both the common register and the private register corresponding to the first thread bundle simultaneously.

[0075] In some embodiments, after the first thread bundle completes the execution of all first-stage instructions, the processor unit generates at least two types of second thread bundles to execute second-stage instructions. The at least two types of second thread bundles can improve computational efficiency by executing second-stage instructions asynchronously and in parallel.

[0076] The number of the aforementioned second thread bundles can be defined in the first-stage instructions and the second-stage instructions, and the function or category of each second thread bundle can be defined in the second-stage instructions.

[0077] In some embodiments, the first-stage instruction can be obtained by acquiring the instruction address corresponding to the matrix multiplication task; correspondingly, the address of the second-stage instruction can be determined based on the instruction address and the instruction offset, thereby acquiring the second-stage instruction. Here, the instruction offset can be determined based on the address difference between the address of the first-stage instruction and the address of the second-stage instruction.

[0078] In this embodiment, by responding to the matrix multiplication task being assigned to the processor unit and allocating a target register for the matrix multiplication task in the processor unit, not only can the register resources be initialized, but also different types of second thread bundles can be generated to use the same common data and store it in a common register during the execution of the matrix multiplication task. This effectively avoids the resource consumption caused by each type of second thread bundle needing to generate the common data.

[0079] In some embodiments, the target register includes a private register, which includes a first sub-register corresponding to a first thread bundle, or a second sub-register corresponding to each second thread bundle; each second sub-register is not visible to other thread bundles. The target register includes a private register corresponding to each thread bundle; a private register corresponding to a thread bundle is not visible to other thread bundles.

[0080] In some embodiments, the target register may include a first sub-register corresponding to a first thread bundle, and may also include a second sub-register corresponding to each second thread bundle. Private registers between different thread bundles are mutually invisible; for example, a first thread bundle cannot access the second sub-register corresponding to a second thread bundle, and a second thread bundle cannot access the first sub-register corresponding to a first thread bundle. For different types of second thread bundles, a second thread bundle of type A cannot access the second sub-register corresponding to a second thread bundle of type B, and vice versa.

[0081] In some embodiments, second sub-registers may be visible to each other between different second thread bundles of the same category. In the current embodiment, different thread bundles of the same category can operate on the same registers through synchronization mechanisms (such as barrier instructions).

[0082] In other embodiments, the second sub-registers may be invisible to each other between different second thread bundles of the same category.

[0083] For example, in order to improve the computational efficiency of matrix multiplication tasks, two or more threads can be set for the second thread bundle that implements the computation process, thereby achieving the effect of at least two thread bundles executing the computation stage in parallel.

[0084] In the embodiments of the present application, by setting shared registers and private registers in the target register, where the shared registers are visible to various warps, it avoids the frequent transfer of multiply-accumulate results between the on-chip shared buffer and the warp private registers, thus saving on-chip bandwidth; while the private registers enable each warp to have an independent storage space for storing its own private data and temporary variables, reducing data competition and conflicts, making the calculations between warps independent of each other, and improving the stability and reliability of the program.

[0085] In some embodiments, the matrix multiplication task is executed in blocks, calculating the product between a first partial matrix in the first matrix and a second partial matrix in the second matrix to obtain a target matrix block; the first warp is used to calculate the position of the target matrix block in the third matrix and store it in the shared register; the third matrix is the product of the first matrix and the second matrix.

[0086] Among them, the matrix multiplication task refers to the operation process of generating a third matrix by linearly combining the corresponding elements of two matrices. The above first warp can calculate the position of the target matrix block in the third matrix through the first-stage instruction.

[0087] The above-mentioned matrix multiplication task being executed in blocks means that during the execution of the matrix multiplication between the first matrix and the second matrix, each part in the third matrix is calculated in a block manner, and then the final third matrix can be obtained; here, the part of the third matrix is the target matrix block. Correspondingly, during the process of calculating the target matrix block, it is also obtained by calculating the product between the first partial matrix in the first matrix and the second partial matrix in the second matrix; the first partial matrix in the first matrix here is also a part of the first matrix, and the second partial matrix in the second matrix is also a part of the second matrix.

[0088] Please refer to Figure 1C , the first matrix (A) and the second matrix (B) can perform matrix multiplication to obtain the third matrix (C). To improve the calculation efficiency, the third matrix C can be divided into multiple sub-blocks, where the target matrix block is Figure 1C C_tile in ; correspondingly, under the block matrix multiplication strategy, the first partial matrix can be the sub-block divided from the original first matrix. For example, if the first matrix is A (dimension M×K), its first partial matrix can be (dimension n×k, n < N, k < K); the second partial matrix is the sub-block divided from the second matrix. For example, if the second matrix is B (dimension K×N), its second partial matrix can be (dimension k×m, m < M), so that the products of multiple sub-blocks can be accumulated to obtain .

[0089] In some embodiments, the position of the target matrix block within the third matrix is ​​used at least to determine the matrix elements of the target matrix block within the third matrix. For example, if the third matrix is ​​divided into multiple sub-blocks according to a partitioning strategy, each sub-block has a position index, so that... Figure 1C In For example, the position index of the target matrix block can include i and j; if m×n is the block size, the starting position of the target matrix block in the third matrix can be (i×m, j×n).

[0090] In some embodiments, the position index of the target matrix block can be stored in the target register, or the starting position of the target matrix block in the third matrix can be stored in the target register. Subsequent second thread bundles can determine the first and second parts of the matrix to be read based on the position of the target matrix block in the third matrix, or, during block multiplication and accumulation, determine the first part of the matrix to be read in each multiplication and accumulation process. and from the second part of the matrix It can also determine the location of the multiplication and accumulation result in the target register during the multiplication and accumulation process; it can also determine the memory address of the target matrix block in the system memory.

[0091] In this embodiment, the position of the target matrix block in the third matrix is ​​calculated by the first thread beam, and the position information is stored in the target register, which can provide an accurate position reference for subsequent calculation processes. At the same time, it can enable at least two subsequent types of second thread beams to correctly read, process and store data based on common position information.

[0092] In some embodiments, the matrix multiplication task includes at least the following subtasks: a loading task, an accumulation task, and a storage task; a second thread bundle is used to execute at least one of the subtasks, and different second thread bundles correspond to different subtasks.

[0093] In this embodiment, the matrix multiplication task can be divided into multiple task stages, with each task stage serving as a subtask. By assigning at least one subtask to a type of second thread bundle, the resulting multiple subtasks can be assigned to different second thread bundles for asynchronous parallel execution, thus fully utilizing the computing power of parallel computing resources.

[0094] Among them, the loading task mentioned above refers to the task used to read matrix elements from system memory into the computing unit during matrix multiplication, which can be understood as the data preparation stage; the accumulation task is used to perform the accumulation operation after the multiplication operation to obtain an element value of the result matrix; the storage task refers to the task used to write the calculated result matrix elements back to system memory after the matrix multiplication is completed.

[0095] In some embodiments, the loading task is used to read elements of the first matrix and the second matrix from system memory; the accumulation task is used to perform multiplication and accumulation operations on matrix elements to obtain elements of the third matrix; and the storage task is used to write the calculated elements of the third matrix back to memory.

[0096] In some embodiments, a type of second thread bundle is used to execute at least one of the subtasks. That is, each second thread bundle can be used to execute one subtask, or one type of second thread bundle can be used to execute two subtasks, and another type of second thread bundle can be used to execute one subtask.

[0097] For example, a second thread bundle can be assigned to load tasks, a second thread bundle can be assigned to accumulate tasks, and a third thread bundle can be assigned to store tasks.

[0098] For example, one second thread bundle is responsible for loading tasks, and a second second thread bundle is assigned to handle accumulating and storing tasks; or, one second thread bundle is assigned to handle loading and accumulating tasks, and a second second thread bundle is assigned to handle storing tasks; or, one second thread bundle is assigned to handle loading and storing tasks, and a second second thread bundle is assigned to handle accumulating tasks.

[0099] In the embodiments of this application, by decomposing the matrix multiplication task into smaller subtasks and assigning them to different second thread bundles for parallel execution, the total execution time of the task can be reduced and the computational throughput can be improved.

[0100] In some embodiments, the second thread bundle executing the loading task is used to load the first partial matrix and the second partial matrix from system memory into the on-chip cache; wherein, the private register corresponding to the second thread bundle executing the loading task is used to store the first memory address of the first partial matrix and the second memory address of the second partial matrix.

[0101] The on-chip cache mentioned above is a high-speed storage unit integrated inside the processor unit, with an access speed much higher than that of system memory. For example, it can be a shared register or L1 cache within the processor unit. System memory is the computer's main memory (such as Dynamic Random Access Memory, DRAM), used to store the input matrix (first matrix, second matrix) and output matrix (third matrix) of matrix multiplication.

[0102] In some embodiments, the second thread bundle performing the loading task can read the first part of the matrix from the system memory based on the first memory address of the first part of the matrix and load it into the on-chip cache; and can read the second part of the matrix from the system memory based on the second memory address of the second part of the matrix and load it into the on-chip cache.

[0103] It should be noted that the second thread bundle executing the loading task can also obtain the aforementioned first memory address and second memory address, and store them in the private register corresponding to the second thread bundle executing the loading task.

[0104] In some embodiments, after each load is completed, the second thread bundle executing the load task can update the memory address in the private register (e.g., increment the offset); it can also notify the second thread bundle executing the accumulation task that the data is ready via a barrier instruction / synchronization instruction.

[0105] In some embodiments, the on-chip addresses of the first part matrix and the second part matrix in the on-chip cache can be pre-set based on preset rules.

[0106] In the above embodiments, by utilizing the parallel processing capability of thread bundles and the method of storing memory addresses in private registers, the transmission of matrix data from system memory to on-chip cache can be achieved efficiently.

[0107] In some embodiments, the second thread bundle executing the accumulation task is used to load the first part matrix and the second part matrix from the on-chip cache to perform matrix element multiplication and accumulation; wherein, the private register corresponding to the second thread bundle executing the accumulation task is used to store the first on-chip address of the first part matrix and the second on-chip address of the second part matrix.

[0108] In some embodiments, the second thread bundle performing the accumulation task can read the first part of the matrix from the on-chip cache based on the first on-chip address of the first part of the matrix, and read the second part of the matrix from the on-chip cache based on the second on-chip address of the second part of the matrix, and then perform the matrix element multiplication and accumulation process to obtain the multiplication and accumulation result of the current loop (kth iteration), and store the multiplication and accumulation result in a shared register.

[0109] It should be noted that the second thread bundle executing the accumulation task can also obtain the first on-chip address and the second on-chip address mentioned above, and store them in the private register corresponding to the second thread bundle executing the accumulation task. As mentioned earlier, the first on-chip address of the first part matrix and the second on-chip address of the second part matrix can be pre-set based on preset rules. Therefore, after the loading task stores the two part matrices in the on-chip cache, the accumulation task can find the two part matrices based on the same on-chip address.

[0110] In some embodiments, after each multiplication-accumulation is completed, the second thread bundle performing the accumulation task can update the on-chip address in the private register (e.g., increment the offset), thereby finding the first and second part matrices corresponding to the next loop loaded on the on-chip buffer.

[0111] In some embodiments, after completing the Kth iteration, the calculation of the target matrix block has been completed, and the second thread bundle executing the storage task can be notified via a barrier instruction / synchronization instruction that the data is ready.

[0112] In some embodiments, the position of the multiplication-accumulation result in the shared register is predetermined based on a preset rule.

[0113] In some embodiments, the intermediate result includes a multiply-accumulate result, and the common register includes an accumulation register; the accumulation register is used to store the multiply-accumulate result generated by the second thread bundle that performs the accumulation task, and the first thread bundle is also used to initialize the accumulation register in the target register.

[0114] The first thread can initialize the accumulator register using the first-stage instructions, setting it to a known initial value (such as 0). Then, when the second thread performs the accumulation task, it adds the calculation result to the accumulator register. In this way, the accumulator register can continuously store and update the multiplication-accumulation result until the Kth iteration is completed.

[0115] In the above embodiments, by using a shared register design, the utilization rate of registers and computational efficiency can be improved, while the overhead of parallel computing can be reduced.

[0116] In some embodiments, the second thread bundle executing the storage task is used to write the multiply-accumulate result in the shared register to the system memory; wherein, the private register corresponding to the second thread bundle executing the storage task is used to store the third memory address of the target matrix block.

[0117] In some embodiments, the second thread bundle performing the storage task can accumulate the position of the register in the shared register, read the multiply-accumulate result that has been calculated from the shared register, and write it to the system memory.

[0118] Furthermore, the second thread executing the storage task can further post-process the multiply-accumulate result after obtaining it, including at least one of the following: scaling, offsetting, etc. Scaling refers to multiplying the multiply-accumulate result by a scaling factor to adjust its numerical range. Offset refers to adding an offset to the scaled result to further adjust the result value. These two steps can be combined into a linear transformation formula: D = ACC * scaling + offset, where D is the post-processed result (i.e., the target matrix block), ACC is the multiply-accumulate result, scaling is the scaling factor, and offset is the offset amount. Finally, the second thread executing the storage task can write the post-processed result to system memory.

[0119] It should be noted that the second thread bundle executing the storage task can also obtain the aforementioned third memory address and store it in the private register corresponding to the second thread bundle executing the storage task. In some embodiments, the private register corresponding to the second thread bundle executing the storage task can also be used to store parameters used in the post-processing process, such as scaling factors and offsets.

[0120] In some embodiments, the third memory address may be pre-defined based on preset rules and is related to the position of the target matrix block in the third matrix and the memory address of the third matrix in the system memory.

[0121] In the above embodiments, efficient writing of multiplication-accumulation results from a shared register to system memory can be achieved.

[0122] It should be noted that the second thread bundle for executing the loading task, the second thread bundle for executing the computation task, and the second thread bundle for executing the storage task can be different second thread bundles or the same second thread bundle. If they are the same second thread bundle, it means that the second thread bundle can execute at least two subtasks simultaneously. For example, if the second thread bundle for executing the loading task and the second thread bundle for executing the computation task are the same second thread bundle, it means that the second thread bundle can execute both the loading task and the computation task simultaneously.

[0123] In some embodiments, the target register is a physical space of a preset size; the preset size is determined based on a first size of the shared register and a second size of the private register corresponding to each thread bundle.

[0124] In some embodiments, the target register may simultaneously include the private register corresponding to the first thread bundle and the private register corresponding to the second thread bundle. That is, when the second thread bundle begins execution, although the first thread bundle does not exist at that time, the target register still contains the private register of the first thread bundle. Therefore, the preset size is determined based on the first size of the shared register, the second size of the private register of the first thread bundle, and the second size of the private register corresponding to each of the second thread bundles. For example, it can be the sum of the first size of the shared register, the second size of the private register of the first thread bundle, and the second size of the private register corresponding to each of the second thread bundles.

[0125] In some embodiments, the target register is a physical space of a preset size; the preset size is determined based on a first size of the shared register and a second size of the private register corresponding to each of the second thread bundles.

[0126] In some embodiments, the preset size is determined based on a first size of the common register and a second size of the second sub-register corresponding to each of the second thread bundles.

[0127] The preset size can be a size pre-set for the target register during the design or configuration phase. This preset size is determined based on a first size of the shared register and a second size of the private registers (second sub-registers) of each of the second thread bundles to ensure that the target register can accommodate all the necessary data.

[0128] In this embodiment, since the first thread bundle and the second thread bundle belong to two sequentially executed phases, and considering that the second size of all second thread bundles is greater than the size of the private register of the first thread bundle, only the second size of the private register corresponding to the second thread bundle can be considered when designing the preset size of the target register. That is, a portion of the non-shared registers in the target register serves as the private register of the first thread bundle during its execution, and serves as the private register corresponding to the second thread bundle during its execution. In other words, in the target register, there exists a private register that is reused by both the first and second thread bundles; when the first thread bundle is generated, this private register is the first sub-register of the first thread bundle; when the second thread bundle is generated, this private register is the second sub-register of the second thread bundle.

[0129] In some embodiments, the logical address range of the first thread bundle is the sum of the logical address range of the common register and the logical address range of the first sub-register; the logical address range of the first thread bundle is the sum of the logical address range of the common register and the logical address range of the corresponding second sub-register.

[0130] The logical address range of a thread bundle consists of two parts: a first size of shared registers (public space) and a second size of private registers (private space). Accordingly, the logical address range of a thread bundle can be determined as follows: Logical address range = start address to end address of shared registers + start address to end address of private registers.

[0131] For example, if the public register occupies 0x0000-0x0FFF (4KB) and the private register of the first thread bundle or the second thread bundle occupies 0x1400-0x17FF (1KB), then the logical address range of the first thread bundle or the second thread bundle is 0x0000-0x13FF (5KB). The size of the register space is only an example to illustrate the calculation method of the logical address range and is not a limitation on the embodiments of this application.

[0132] In some embodiments, the second size of the private registers for different thread bundles may be the same or different. In other embodiments, the second size of the private registers corresponding to the first thread bundle and the second thread bundle may be the same or different. In still other embodiments, the second size of the private registers corresponding to different second thread bundles may be the same or different.

[0133] In some embodiments, the second size of the second sub-registers corresponding to each of the second thread bundles is the same; and / or, the second size of any of the second sub-registers is the same as the third size of the first sub-register.

[0134] In some possible implementations, the second sub-registers corresponding to each second thread bundle have the same size. That is, the second sub-registers corresponding to different second thread bundles are of the same size. For example, if multiple second thread bundles (such as thread bundle A, thread bundle B, and thread bundle C) participate in the operation simultaneously, each thread bundle has a corresponding second sub-register to store data. Assume the second sub-register size corresponding to thread bundle A is 128 bytes, the second sub-register size corresponding to thread bundle B is also 128 bytes, and the second sub-register size corresponding to thread bundle C is also 128 bytes; that is, the second sub-register sizes corresponding to all second thread bundles are the same.

[0135] In some other possible implementations, the second size of any second sub-register is the same as the third size of the first sub-register. That is, the size of all second sub-registers is the same as the size of the first sub-register. Suppose there is a first sub-register whose third size is set to 256 bytes, and there are multiple second sub-registers (such as second sub-register X, second sub-register Y, and second sub-register Z), where the size of second sub-register X is 256 bytes, the size of second sub-register Y is also 256 bytes, and the size of second sub-register Z is also 256 bytes, meaning that the size of any second sub-register is the same as the size of the first sub-register.

[0136] It is understandable that the two possible implementation methods described above can be satisfied simultaneously, that is, the second sub-registers corresponding to each second thread bundle have the same size, and the size of any second sub-register is the same as the size of the first sub-register; or only one of the conditions can be satisfied.

[0137] In other embodiments, there exists a second size of the second sub-register that is the same as the third size of the first sub-register.

[0138] Here, the second sub-register, with the same third size as the first sub-register, is a private register shared by the first and second thread bundles. If a first sub-register exists with a third size set to 256 bytes, at least one of the multiple second sub-registers must also be 256 bytes in size. For example, there may be multiple second sub-registers (such as second sub-register X, second sub-register Y, and second sub-register Z), where second sub-register X is 256 bytes, second sub-register Y could be 512 bytes, and second sub-register Z could be 128 bytes. The sizes of these different second sub-registers can be related to the functionality of the corresponding second thread bundle.

[0139] In some embodiments, the logical address range of each thread bundle is the sum of the logical address range of the common register and the logical address range of the corresponding private register.

[0140] The second size of the private register corresponding to the first thread bundle and the second size of the private register corresponding to each second thread bundle are the same.

[0141] In this embodiment, all thread bundles share the same logical address range. Although each thread bundle's private registers are physically independent (e.g., thread bundle A's private registers are in the range of 0x1000-0x13FF, and thread bundle B's are in the range of 0x1400-0x17FF), the logical addresses are uniformly mapped in the following way: Shared registers: the logical addresses of all thread bundles from 0x0000 to 0x0FFF are mapped to the same physical shared registers; Private registers: the logical addresses of each thread bundle from 0x1000 to 0x13FF are mapped to its own dedicated physical private registers (e.g., thread bundle A is mapped to 0x1000-0x13FF, and thread bundle B is mapped to 0x1400-0x17FF). This design allows thread bundles to access data without being aware of physical address differences, simply by using the same logical address range.

[0142] For example, the logical addresses 0x0000-0x13FF of thread bundle A correspond to physical addresses 0x0000 (shared register) and 0x1000 (Wave0 private register); the logical addresses 0x0000-0x13FF of thread bundle B correspond to physical addresses 0x0000 (shared register) and 0x1400 (Wave1 private register). Thread bundle A and thread bundle B can be understood as either the first thread bundle or the second thread bundle.

[0143] In this embodiment, by setting the second size of the private registers corresponding to each thread bundle to be the same, it can be ensured that each thread bundle has the same amount of private storage space, which facilitates unified management and allocation. At the same time, by setting the logical address range of all thread bundles to the sum of the logical address range of the shared register and the logical address range of the corresponding private register, and mapping them in a unified manner, different thread bundles do not need to care about the underlying physical address differences when accessing data. They can access the required data only through the same logical address range. The programming model does not need to adjust the address offset for different thread bundles, reducing code complexity.

[0144] During the process of the first thread beam accessing the target register, the logical address requested by the first thread beam can be mapped to the actual physical address through a logical address translation process. In some embodiments, the method further includes: determining the target physical address corresponding to the logical address requested by the first thread beam based on the physical base address and logical address offset of the shared register.

[0145] In some embodiments, for a first thread bundle, the target register includes a common register and a private register corresponding to the first thread bundle. The physical base address of the private register of the first thread bundle is immediately adjacent to the end address of the common register (for example, the common register ends at 0x0FFF, and the private register of the first thread bundle starts at 0x1000).

[0146] In some embodiments, the logical address offset is the offset of the logical address relative to the logical base address. During address translation, the logical address offset is actually the same as the physical address offset. Therefore, when the physical base address of the private register of the first thread bundle is adjacent to the end address of the shared register, the target physical address can be determined directly based on the sum of the physical base address of the shared register and the logical address offset.

[0147] For example, assume the physical base address of the shared register is fixed at 0x0000, and the physical base address of the private register of the first thread bundle immediately follows (0x1000), contiguous with the shared register. The logical address range of the first thread bundle is defined as 0x0000-0x13FF, covering both the shared register (0x0000-0x0FFF) and the private register of the first thread bundle (0x1000-0x13FF). During address translation, logical addresses 0x0000-0x0FFF are directly mapped to the physical addresses 0x0000-0x0FFF of the shared register; logical addresses 0x1000-0x13FF are mapped to the physical addresses 0x1000-0x13FF of the private register of the first thread bundle. Accordingly, the target physical address = the physical base address of the shared register + the logical address offset, without the need for table lookup or dynamic calculation.

[0148] In this embodiment, by determining the target physical address corresponding to the logical address of the first thread bundle request based on the physical base address and logical address offset of the shared register, a fast mapping from logical address to physical address can be achieved, improving the efficiency of address translation. At the same time, this mapping method simplifies the complexity of address management, enabling the system to handle thread bundle access requests to registers more flexibly.

[0149] Similar to the first thread bundle, during the second thread bundle's access to the target register, a logical address translation process can be used to map the logical address requested by the second thread bundle to an actual physical address. In some embodiments, the target register is a contiguous physical space, and the method further includes: if the logical address requested by the second thread bundle is less than the maximum logical address of the shared register, determining the target physical address corresponding to the logical address requested by the second thread bundle based on the physical base address and logical address offset of the shared register; if the logical address requested by the second thread bundle is greater than or equal to the maximum logical address of the shared register, determining the target physical address corresponding to the logical address requested by the second thread bundle based on the physical base address of the shared register, the identifier of the thread bundle, the logical address range of the second sub-register, and the address offset.

[0150] In some embodiments, the private registers (i.e., second sub-registers) corresponding to each second thread bundle are physically arranged consecutively. That is, the second sub-registers are arranged consecutively according to the identifier of the second thread bundle, the end address of the nth second sub-register is adjacent to the start address of the (n+1)th second sub-register, and the second thread bundle corresponding to the nth second sub-register is the previous second thread bundle corresponding to the (n+1)th second sub-register.

[0151] In some embodiments, when the logical address requested by the second thread bundle is less than the maximum logical address of the shared register, it indicates that the second thread bundle needs to access the shared register; when the logical address requested by the second thread bundle is greater than or equal to the first maximum value of the shared register, it indicates that the second thread bundle needs to access the private register.

[0152] For example, assume the physical base address of the shared register is 0x0000, the maximum logical address is 0x1000 (i.e., 4KB), the logical address range of the private register starts from 0x1000, and each second thread bundle has an independent private register range. If the logical address requested by the second thread bundle is 0x0500, then the target physical address (0x0000 + 0x0500 = 0x0500) is directly calculated based on the physical base address of the shared register and the logical address offset (0x0500).

[0153] For example, if the logical address requested by the second thread bundle is 0x1200, it indicates that it expects to access a private register. Therefore, the identifier of the second thread bundle can be read first, and the physical base address of the private register can be calculated. Physical base address = common register base address + maximum logical address + (thread bundle identifier) 1) × Second size of private register. For example, if the maximum logical address is 4KB (0x1000) and the second size of private register is 1KB (0x400), then the physical base address of the private register corresponding to the first second thread bundle = 0x0000 + 0x1000 + (1-1) * 0x400 = 0x1000; at the same time, logical address offset = logical address - maximum logical address. For example, if the logical address is 0x1200, the logical address offset = 0x1200 - 0x1000 = 0x0200; the target physical address = physical base address of the private register corresponding to the first second thread bundle (0x1000) + logical address offset (0x0200) = 0x1200.

[0154] In this embodiment, efficient mapping from the logical address to the physical address of the second thread beam request can be achieved. When the logical address is within the range of the shared register, mapping is performed directly using the physical base address and the logical address offset, simplifying the address calculation process. When the logical address exceeds the range of the shared register, mapping is performed by combining the second thread beam identifier and the range of the private register, ensuring the accuracy and flexibility of the address mapping.

[0155] In some embodiments, the method further includes: reserving the shared register in response to the first thread bundle completing all instruction execution.

[0156] During the execution of the first stage instructions by the first thread bundle, it can implement its own task function through the private register corresponding to the first thread bundle, and pass the common data (e.g., the position of the target matrix block in the third matrix) required by each of the second thread bundles when executing the second stage instructions through the shared register.

[0157] In some embodiments, since the shared register is visible to all thread bundles (the first thread bundle and at least two types of second thread bundles), it is necessary to retain the shared register even after the first thread bundle has completed all first-stage instruction execution.

[0158] In some embodiments, the private register corresponding to the first thread bundle can be directly allocated to the subsequent second thread bundle; in other embodiments, the private register corresponding to the first thread bundle can be initialized first and then allocated to the subsequent second thread bundle. It is understood that since the target register is allocated to the matrix multiplication task, the private register should not be released when the first thread bundle completes all first-stage instruction execution, even if the data stored in the private register corresponding to the first thread bundle is no longer needed.

[0159] In the above embodiments, by retaining the shared register after the first thread bundle completes all instruction execution, the continuous availability of the common data can be maintained, enabling the second thread bundle that depends on the data to run normally, reducing the risk of calculation errors caused by premature data release, while reducing the overhead of repeatedly allocating and loading common data, and improving the execution efficiency of parallel tasks.

[0160] In some embodiments, the method further includes: releasing the first sub-register when the task execution of the first thread bundle is completed.

[0161] Here, after the first thread bundle completes its task and before generating the second thread bundle, the first sub-register of the first thread bundle needs to be released. It is understood that the shared register corresponding to the first thread bundle remains unchanged, and the data stored in the shared register also remains unchanged. This way, the second thread bundle generated subsequently can use the data generated by the first thread bundle.

[0162] In this embodiment of the application, the released target register includes the second sub-register corresponding to each second thread bundle.

[0163] In some embodiments, the method further includes: releasing the target register in response to each of the second thread bundles having completed all instruction execution.

[0164] In some embodiments, the processor unit may continuously monitor the execution status of all second thread bundles. When it is confirmed that all second thread bundles have completed all instruction execution, the target register is marked as available so that subsequent thread bundles can reallocate and use these registers.

[0165] In the above embodiments, by releasing the target register after all second thread bundles have completed instruction execution, physical storage resources can be reclaimed in a timely manner, the occupation of idle resources can be reduced, the utilization rate of registers can be improved, and the possibility of data overwriting due to incomplete operations can be reduced by the detection mechanism based on instruction commit status, thereby enhancing the dynamic adaptability of resource management.

[0166] In multi-task parallel scenarios, the first thread bundle of a new task may preempt computing resources, causing the second thread bundle of already started tasks to be blocked due to insufficient resources, thus reducing system throughput. Therefore, this application assigns the highest priority to the generation of the second thread bundle, ensuring that the computational operations of already started tasks receive resources first; the generation of the first thread bundle of a new task must wait for the completion of the second thread bundle of the current task (completion of the second phase), thereby guaranteeing the efficient execution of already started tasks. Please refer to [link to relevant documentation]. Figure 5 , Figure 5 This is a schematic diagram of the implementation flow of a matrix multiplication task execution method provided in an embodiment of this application. Figure 2 This method can be executed by the processor of a computer device. The generation priority of the at least two types of second thread bundles is higher than the generation priority of the first thread bundle. The method may further include steps S501 to S502, combining... Figure 5 The steps shown are explained.

[0167] Step S501: In response to a new matrix multiplication task being assigned to the processor unit and the first thread bundle of the matrix multiplication task completing all instruction execution, at least two types of second thread bundles for the matrix multiplication task are generated.

[0168] In this scenario, when a new matrix multiplication task is assigned to a processor unit and its first thread bundle has completed instruction execution, a high-priority second thread bundle (such as data loading or multiply-accumulate calculation) is generated first for the second stage of the already started task. In some embodiments, the generation of the first thread bundle for a new matrix multiplication task may require waiting for idle resources or a low-priority time period.

[0169] Step S502: After generating the at least two types of second thread bundles, generate a new first thread bundle; the new first thread bundle is used at least to allocate a new target register for the new matrix multiplication task in the processor unit.

[0170] Specifically, after the second thread bundle of the current task is completed, the first thread bundle of a new matrix multiplication task is generated. This first thread bundle can be used to reclaim the previous target register and allocate a new target register for the new matrix multiplication task.

[0171] Based on the embodiments provided in this application, the system throughput in multi-task parallel scenarios can be effectively improved, ensuring the efficient execution of started tasks and supporting the smooth start of new tasks, thereby enhancing the processor unit's ability to handle complex computing tasks and overall computing performance.

[0172] The following describes the application of the matrix multiplication task execution method provided in the embodiments of this application in a real-world scenario.

[0173] This application proposes an asynchronous programming model and architecture for multi-threaded waves, which allows multiple waves to execute different code segments, each with its own independent space while sharing some register file data, thus saving total resource consumption and avoiding data migration.

[0174] In this embodiment of the application, when a task is launched to the Processor, it can be divided into multiple stages, including: a first stage and a second stage.

[0175] The first phase is used to execute common instructions, such as obtaining the global tile ID and initializing the accumulator register. When the first phase ends, the scheduler returns and N new wave threads are generated. At the same time, a portion of the common register stack is reserved as a common register for the first and second phases. This common register stack is visible to all N wave threads simultaneously.

[0176] Each wave in the second phase has an additional private register. The total register range that each second-phase wave can use is the sum of the common register size and the private register size. The common register is visible to multiple waves simultaneously, and the programmer ensures synchronization (i.e., the producer-consumer dependency relationship and avoids multiple waves updating the common register at the same time).

[0177] The first stage mentioned above is used to calculate the global block identifier of the tile, initialize the accumulator register, and define the number of subsequent waves.

[0178] In the second stage, the functionality of different thread bundles can be distinguished by defining stage2_spawn_id.

[0179] When `stage2_spawn_id` is 0, it indicates that a loading task is executed, which moves the blocks of matrices A and B to shared memory. After loading is complete, a computation task is triggered. During the execution of the loading task, it is necessary to load the blocks to be computed from system memory. Therefore, the block information of the blocks to be computed is required. The block information can include the address and size of the blocks in system memory (global memory) of matrix A, and the address and size of the blocks in system memory of matrix B. Here, the above block information can be stored in the private registers corresponding to the loading task.

[0180] When `stage2_spawn_id` is 1, it indicates that a computation task is being executed. After the accumulation process is completed, a storage task is triggered. During the execution of the computation task, the block to be computed needs to be loaded from the on-chip cache. Therefore, the block information of the block to be computed is required. The block information can include the address of the block in matrix A in the on-chip cache, and the address of matrix B in the on-chip cache. Here, the above block information can be stored in the private register corresponding to the computation task.

[0181] When stage2_spawn_id is 2, it indicates that a storage task is being executed. During the execution of the storage task, the final multiply-accumulate result stored in the register needs to be stored in system memory. Therefore, the address of the multiply-accumulate result in system memory is needed, and this address can be stored in the private register corresponding to the storage task.

[0182] Please see Figure 6 It shows a schematic diagram of the execution flow of a thread bundle.

[0183] Step S61: Task Startup.

[0184] Step S62: Thread bundle generation.

[0185] Different thread bundles need to be generated at different stages of task execution. Step S62 may include:

[0186] Step S621: When the task starts and is in the first stage, generate a thread bundle.

[0187] Here, a thread bundle can be understood as the first thread bundle in the above embodiment.

[0188] Step S622: In the second stage, generate N thread bundles.

[0189] The N thread bundles here can be understood as at least two types of second thread bundles in the above embodiments.

[0190] In some embodiments, the generated thread bundle can begin execution based on the following address: the new task instruction base address (first thread bundle), instruction base address + offset of the second stage (second thread bundle).

[0191] Step S63: Thread bundle execution.

[0192] Step S64: Is it in the second stage?

[0193] Specifically, if the process is in the second stage, step S65 is executed; if the process is not in the second stage, step S66 is executed.

[0194] Step S65: Continue execution.

[0195] In this case, considering that there are N second thread bundles in the second phase, it is necessary to wait for each second thread bundle to finish executing.

[0196] Step S66: The current thread bundle ends, but the registers are not released.

[0197] In this case, considering that there is only one first thread bundle in the first phase, the previously allocated target register does not need to be released after the execution of the current first thread bundle ends, so as to continue to provide register resources for the subsequent second thread bundle.

[0198] In some embodiments, after the first thread bundle ends, a new thread bundle is generated, and the address of the instruction to be executed is (instruction base address + offset of the second stage).

[0199] Step S67: Thread bundle ends. Release the threads after all generated thread bundles have ended.

[0200] Specifically, the calculation of a matrix block is only complete and written to system memory after all thread bundles have finished, at which point the previously allocated target register can be released.

[0201] In some embodiments, the above process can be accomplished by a scheduler located in the processor unit. This scheduler may include a task manager and a wave generator. It needs to support wave launches for new tasks and wave launches for tasks that have already reached the second stage (2nd stage task). To ensure the priority of already executed tasks, tasks that have reached the second stage always have the highest wave generation priority.

[0202] The second thread bundle generation requires dividing the original task into N thread bundles based on the number of thread bundles specified in the second stage instruction, and synchronously updating the instruction address of the thread bundle to base_pc + stage2_pc_offset.

[0203] In some embodiments, the processor unit may further include a task termination module, which releases registered resources uniformly after all the second thread bundles in the second phase have finished executing. The number of second thread bundles that have finished executing can be counted using an object counter (spawncounter).

[0204] Please see Figure 7 This illustrates a register allocator architecture. During the first phase of a new task's request, a contiguous space of common_size + private_size * N is allocated uniformly. Here, common_size is the size of the shared registers, private_size is the size of the private registers, and N is the number of private registers, or the number of second thread bundles.

[0205] For the first stage, the parent wave of the first stage, i.e. the first thread wave in the above embodiment, can use a logical space range of common_size + private_size, and the physical space is mapped to the space range of the first thread wave of the first stage.

[0206] For the second phase, the logical space range that the child wave in the second phase can use is common_size + private_size. The physical space is divided into two segments: the first segment is the common space, and the second segment is the private space of each child wave.

[0207] In some embodiments, the hardware address calculation unit is responsible for converting logical addresses to physical addresses. When the logical address (VA) is less than the preset size of the common register (common_size), it is determined that the address is located in the common register. In this case, the physical address (PA) is calculated using the following formula: PA = Common register base address (common_base) + Logical address offset (va_offset). If the logical address exceeds the size of the common register, the physical address calculation needs to incorporate the sub-thread bundle identifier (n) and the size of the private register (private_size), and the calculation formula is: PA = Common register base address (common_base) + Total private register capacity (private_size × (n-1)) + Logical address offset (va_offset).

[0208] In the second stage of the judgment, the logical address of the access register is used to determine whether it is in a shared range or a private range of each sub-wave, and then it is mapped to the physical address.

[0209] In some embodiments, for a workgroup (a logical level where multiple waves work together, such as a workgroup in OpenCL and a block in CUDA), there are producer-consumer relationships between different waves, requiring synchronization between waves. For example, the computation wave needs to wait for the data loading wave to move all the data into the on-chip cache, and the post-processing store wave needs to wait for the computation wave to complete the computation.

[0210] In some embodiments, a barrier instruction (work-group barrier) can be used to achieve inter-process synchronization between work-items within the same work-group; the barrier instruction is split into two instructions: an arrival instruction and a wait instruction.

[0211] In this embodiment of the application, the aforementioned waves can be synchronized using an async arrive / wait bar.

[0212] When a thread executes a arrival instruction, it atomically decrements the arrival count in the arrival count cache for the current workgroup and the current barrier instruction. It can then continue executing instructions without data dependencies. When a thread executes a wait instruction, it stores its current thread ID in the sleep control cache and then goes to sleep. When the arrival count reaches 0, the sleep control parameters are used to wake up threads that have already executed wait instructions, and the sleep control parameters are cleared to 0. Compared to related schemes, the waiting state is asynchronous, eliminating the need for all threads to reach the waiting state before waking them up.

[0213] Please see Figure 8 , Figure 8 This is a schematic diagram of an asynchronous waiting scheme for threads based on arrival instructions and wait instructions, provided in an embodiment of this application. The current workgroup includes nine threads, from thread 0 to thread 8. Among them, threads 0 to 4 support the first barrier instruction bar0, threads 5 to 7 support the second barrier instruction bar1, and thread 8 does not participate in any barrier instructions (except for the barrier instruction used for initialization).

[0214] Before executing any instruction, an arbitrary thread needs to initialize the arrival count and sleep control parameters corresponding to each barrier instruction. For example, for the first barrier instruction bar0, thread 0 in the diagram can call the first initialization instruction "Initialize 0" to set the arrival count 61 and sleep control parameter 62 of the first barrier instruction bar0 to "5, 0b" respectively; for the second barrier instruction bar1, thread 0 in the diagram can call the second initialization instruction "Initialize 1" to set the arrival count 66 and sleep control parameter 67 of the second barrier instruction bar1 to "3, 0b" respectively. It should be noted that after the initialization instructions for all barrier instructions in the current workgroup, an initialization barrier instruction 65 is needed to ensure that all threads in the current workgroup execute instructions only after initializing the arrival count.

[0215] After all threads have executed the initialization barrier instruction 65, subsequent instructions need to be executed. When the thread corresponding to the current barrier instruction executes the arrival instruction corresponding to the current barrier instruction, it will atomically decrement the arrival count value corresponding to the current barrier instruction. At the same time, it will determine whether to execute the wait instruction based on whether the current instruction is a producer.

[0216] The following embodiments will be illustrated using the first barrier instruction bar0 as an example. Figure 8 As shown, the execution process of each thread corresponding to the first barrier instruction bar0, in chronological order, includes:

[0217] (1) Thread 3 first executes the arrival instruction 63, and decrements the arrival count value 61 by one. At this time, the arrival count value 61 is 4.

[0218] (2) When thread 0 executes the arrival instruction 63, it decrements the arrival count value 61 by one. At this time, the arrival count value 61 is 3.

[0219] For threads 0 and 3 mentioned above, since threads 0 and 3 are producers, there is no need to execute the wait instruction 64, and the sleep control parameter 62 remains unchanged.

[0220] (3) When thread 2 executes the arrival instruction 63, it decrements the arrival count value 61 by one. At this time, the arrival count value 61 is 2.

[0221] (4) When thread 1 executes the arrival instruction 63, it decrements the arrival count value 61 by one. At this time, the arrival count value 61 is 1.

[0222] (5) Since thread 2 is not a producer, it needs to execute the wait instruction 64. Therefore, when thread 2 executes the wait instruction 64, the sleep control parameter 62 is updated to 100b and enters the sleep state.

[0223] (6) Since thread 1 is not a producer, it needs to execute the wait instruction 64. Therefore, when thread 1 executes the wait instruction 64, the sleep control parameter 62 is updated to 110b and enters the sleep state.

[0224] (7) When thread 4 executes the arrival instruction 63, it decrements the arrival count value 61 by one. At this time, the arrival count value 61 is 0. It should be noted that since the arrival count value 61 is 0, the threads in the sleeping state need to be woken up, that is, the threads 1 and 2 in the sleeping state need to be woken up. Threads 1 and 2 continue to execute subsequent instructions.

[0225] (8) Since thread 4 is not a producer, it needs to execute the wait instruction 64. Therefore, when thread 4 executes the wait instruction 64, the sleep control parameter 62 is updated to 10000b and enters the sleep state. Since the count value 61 is 0, thread 4 will be woken up immediately and execute subsequent instructions.

[0226] The following embodiment will be illustrated using the second barrier command bar1 as an example. Figure 8 As shown, the execution process of each thread corresponding to the second barrier instruction bar1, in chronological order, includes:

[0227] (1) Thread 7 first executes the arrival instruction 68, decrements the arrival count value 66 by one. At this time, the arrival count value 66 is 2. Since thread 7 is the producer, it does not need to execute the wait instruction 69. Therefore, the sleep control parameter 67 remains unchanged.

[0228] (2) When thread 5 executes the arrival instruction 68, it decrements the arrival count value 66 by one. At this time, the arrival count value 66 is 1.

[0229] (3) When thread 6 executes the arrival instruction 68, it decrements the arrival count value 66 by one. At this time, the arrival count value 66 is 0.

[0230] (4) Since thread 5 is not a producer, it needs to execute wait instruction 69. When thread 5 executes wait instruction 69, the sleep control parameter 67 is updated to 100000b and it enters a sleep state. At the same time, since the count value 66 is 0, thread 5 will be woken up immediately and execute subsequent instructions. It can be understood that after thread 5 is woken up, the sleep control parameter 67 is updated to 0b (not shown in the figure).

[0231] (5) Since thread 6 is not a producer, it needs to execute wait instruction 69. When thread 6 executes wait instruction 69, the sleep control parameter 67 is updated to 1000000b and it enters a sleep state. At the same time, since the count value 66 is 0, thread 6 will be woken up immediately and execute subsequent instructions. It can be understood that after thread 6 is woken up, the sleep control parameter 67 is updated to 0b (not shown in the figure).

[0232] Based on the above embodiments, by splitting the barrier instruction into an arrival phase and a waiting phase, i.e., splitting it into an arrival instruction and a waiting instruction, the arrival instruction only performs an atomic decrement operation on the arrival count, while the waiting instruction controls the thread to perform a sleep operation to enter a sleep state. Once the arrival count reaches 0, the thread that has already performed the sleep operation can be woken up without waiting for all threads to execute the wait instruction, thus achieving asynchronous barrier instructions. Simultaneously, by adding an initialization arrival instruction, the user can specify the number of threads participating in the barrier instruction. Threads without data dependencies will not be blocked, improving program execution efficiency. Furthermore, a sleep control cache is used to store the thread IDs that need to be waited for, used to wake up the correct thread later. Multiple barrier instructions are supported within the same workgroup, with the specific number specified by the user, which can greatly improve the parallelism in multi-data scenarios.

[0233] Based on the foregoing embodiments, this application provides a matrix multiplication task execution device. The device includes various units and modules included in each unit, which can be implemented by a processor in a computer device; of course, it can also be implemented by specific logic circuits. In the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0234] Figure 9 This is a schematic diagram of the composition structure of a matrix multiplication task execution device provided in an embodiment of this application, as shown below. Figure 9 As shown, the matrix multiplication task execution device 900 includes: an execution module 910 and a generation module 920, wherein:

[0235] Execution module 910 is configured to allocate a target register for the matrix multiplication task in the processor unit during the execution of the matrix multiplication task in the processor unit, wherein the target register includes a shared register;

[0236] The generation module 920 is used to generate at least two types of second thread bundles for performing the matrix multiplication task; the at least two types of second thread bundles asynchronously execute the matrix multiplication task based on the shared register; the shared register is used to store the intermediate results of the matrix multiplication task.

[0237] In some embodiments, the execution module 910 is configured to generate a first thread bundle in response to the matrix multiplication task being assigned to the processor unit; the task executed by the first thread bundle prepares for the execution of the task of the second thread bundle.

[0238] In some embodiments, the target register is allocated by the first thread bundle in the processor unit for the matrix multiplication task, and the shared register is visible to the first thread bundle.

[0239] In some embodiments, the target register includes a private register, which includes a first sub-register corresponding to a first thread bundle, or a second sub-register corresponding to each second thread bundle; each second sub-register is not visible to other thread bundles.

[0240] In some embodiments, the matrix multiplication task is executed in blocks, calculating the product between a first part of the first matrix and a second part of the second matrix to obtain a target matrix block; the first thread is used to calculate the position of the target matrix block in the third matrix and store it in the shared register; the third matrix is ​​the product of the first matrix and the second matrix.

[0241] In some embodiments, the matrix multiplication task includes at least the following subtasks: a loading task, an accumulation task, and a storage task; a second thread bundle is used to execute at least one of the subtasks, and different second thread bundles correspond to different subtasks.

[0242] In some embodiments, the second thread bundle executing the loading task is used to load the first partial matrix and the second partial matrix from system memory into the on-chip cache; wherein, the private register corresponding to the second thread bundle executing the loading task is used to store the first memory address of the first partial matrix and the second memory address of the second partial matrix.

[0243] In some embodiments, the second thread bundle executing the accumulation task is used to load the first part matrix and the second part matrix from the on-chip cache to perform matrix element multiplication and accumulation; wherein, the private register corresponding to the second thread bundle executing the accumulation task is used to store the first on-chip address of the first part matrix and the second on-chip address of the second part matrix.

[0244] In some embodiments, the intermediate result includes a multiply-accumulate result, and the common register includes an accumulation register; the accumulation register is used to store the multiply-accumulate result generated by the second thread bundle that performs the accumulation task, and the first thread bundle is also used to initialize the accumulation register in the target register.

[0245] In some embodiments, when the second thread bundle executing the accumulation task completes, the second thread bundle executing the storage task is used to write the intermediate results in the shared register to the system memory; wherein, the second sub-register of the private register corresponding to the second thread bundle executing the storage task is used to store the third memory address of the target matrix block.

[0246] In some embodiments, the first sub-register is released when the task execution of the first thread bundle is completed.

[0247] In some embodiments, the target register is a physical space of a preset size; the preset size is determined based on a first size of the shared register and a second size of the second sub-register corresponding to each of the second thread bundles.

[0248] In some embodiments, the second size of the second sub-registers corresponding to each of the second thread bundles is the same; and / or, the second size of any of the second sub-registers is the same as the third size of the first sub-register.

[0249] In some embodiments, the logical address range of the first thread bundle is the sum of the logical address range of the common register and the logical address range of the first sub-register; the logical address range of the first thread bundle is the sum of the logical address range of the common register and the logical address range of the corresponding second sub-register.

[0250] In some embodiments, the execution module 910 is configured to determine the target physical address corresponding to the logical address of the first thread beam request based on the physical base address and logical address offset of the shared register.

[0251] In some embodiments, the target register is a contiguous physical space. The execution module 910 is configured to, when the logical address of the second thread bundle request is less than the maximum logical address of the shared register, determine the target physical address corresponding to the logical address of the second thread bundle request based on the physical base address and logical address offset of the shared register; and when the logical address of the second thread bundle request is greater than or equal to the maximum logical address of the shared register, determine the target physical address corresponding to the logical address of the second thread bundle request based on the physical base address of the shared register, the identifier of the thread bundle, the logical address range of the second sub-register, and the address offset.

[0252] In some embodiments, the execution module 910 is configured to retain the shared register in response to the completion of all instruction execution by the first thread bundle.

[0253] In some embodiments, the execution module 910 is configured to release the target register in response to each of the second thread bundles having completed all instruction execution.

[0254] In some embodiments, the generation priority of the at least two types of second thread bundles is higher than the generation priority of the first thread bundle. The execution module 910 is configured to generate at least two types of second thread bundles for the matrix multiplication task in response to a new matrix multiplication task being assigned to the processor unit and the first thread bundle of the matrix multiplication task completing all instruction execution; after generating the at least two types of second thread bundles, a new first thread bundle is generated; the new first thread bundle is used at least to allocate a new target register for the new matrix multiplication task in the processor unit.

[0255] The descriptions of the apparatus embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. In some embodiments, the functions or modules included in the apparatus provided in this application can be used to perform the methods described in the method embodiments above. For technical details not disclosed in the apparatus embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0256] It should be noted that, in the embodiments of this application, if the matrix multiplication task execution method described above is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware, software, or firmware, or any combination of hardware, software, and firmware.

[0257] This application provides a computer device including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements some or all of the steps in the above-described method.

[0258] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements some or all of the steps in the above-described method. The computer-readable storage medium can be transient or non-transient.

[0259] This application provides a computer program including computer-readable code, wherein when the computer-readable code is run in a computer device, a processor in the computer device performs some or all of the steps in the above-described method.

[0260] This application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium; in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0261] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referred to interchangeably. The descriptions of the above embodiments of the device, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0262] Figure 10 This application provides a hardware entity diagram of a computer device as an embodiment of the present application, such as... Figure 10 As shown, the hardware entity of the computer device 1000 includes a processor 1001 and a memory 1002, wherein the memory 1002 stores a computer program that can run on the processor 1001, and the processor 1001 executes the program to implement the steps in the method of any of the above embodiments.

[0263] The memory 1002 stores computer programs that can run on the processor. The memory 1002 is configured to store instructions and applications that can be executed by the processor 1001. It can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data, and video communication data) in the processor 1001 and various modules in the computer device 1000. It can be implemented by flash memory or random access memory (RAM).

[0264] The processor 1001 executes the steps of the matrix multiplication task execution method described above when executing the program. The processor 1001 typically controls the overall operation of the computer device 1000.

[0265] This application provides a computer storage medium storing one or more programs that can be executed by one or more processors to implement the steps of the matrix multiplication task execution method as described in any of the above embodiments.

[0266] It should be noted that the descriptions of the storage medium and device embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0267] The aforementioned processor can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor. It is understood that other electronic devices can also implement the functions of the aforementioned processor, and this application does not specifically limit the specific implementation.

[0268] The aforementioned computer storage media / memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM), etc.; or it can be various terminals that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0269] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above embodiments of this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0270] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0271] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0272] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0273] Furthermore, in the various embodiments of this application, all functional units can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units. Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0274] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, magnetic disks, or optical disks.

[0275] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for performing matrix multiplication tasks, characterized in that, The method, applied to a processor unit in a graphics processor, includes: During the execution of a matrix multiplication task by the processor unit, a target register is allocated for the matrix multiplication task in the processor unit, and the target register includes a shared register; Generate at least two types of second thread bundles for performing the matrix multiplication task; different types of second thread bundles are used to implement different functions; The at least two types of second thread bundles asynchronously execute the matrix multiplication task based on the shared register; the shared register is used to store the intermediate results of the matrix multiplication task; the shared register is visible to at least two types of second thread bundles; the intermediate results include multiply-accumulate results, and the shared register is used to store the multiply-accumulate results.

2. The method according to claim 1, characterized in that, The method further includes: In response to the matrix multiplication task being assigned to the processor unit, a first thread bundle is generated; the task executed by the first thread bundle prepares for the execution of the task of the second thread bundle.

3. The method according to claim 2, characterized in that, The target register is allocated by the first thread bundle in the processor unit for the matrix multiplication task, and the shared register is visible to the first thread bundle.

4. The method according to claim 2, characterized in that, The target register includes a private register, which includes a first sub-register corresponding to a first thread bundle, or a second sub-register corresponding to each second thread bundle; each second sub-register is not visible to other thread bundles.

5. The method according to claim 2, characterized in that, The matrix multiplication task is executed in blocks, calculating the product between the first part of the first matrix and the second part of the second matrix to obtain the target matrix block; The first thread is used to calculate the position of the target matrix block in the third matrix and store it in the shared register; The third matrix is ​​the product of the first matrix and the second matrix.

6. The method according to claim 2, characterized in that, The matrix multiplication task includes at least the following subtasks: Loading tasks, accumulating tasks, and storing tasks; a second thread bundle is used to execute at least one of the subtasks, and different second thread bundles correspond to different subtasks.

7. The method according to claim 6, characterized in that, The second thread bundle that performs the loading task is used to load the first and second parts of the matrix from system memory into the on-chip cache; The private register corresponding to the second thread bundle that executes the loading task is used to store the first memory address of the first part of the matrix and the second memory address of the second part of the matrix.

8. The method according to claim 6, characterized in that, The second thread bundle executing the accumulation task is used to load the first part matrix and the second part matrix from the on-chip cache to perform matrix element multiplication and accumulation; wherein, the private register corresponding to the second thread bundle executing the accumulation task is used to store the first on-chip address of the first part matrix and the second on-chip address of the second part matrix.

9. The method according to claim 8, characterized in that, The shared register includes an accumulation register; the accumulation register is used to store the multiplication and accumulation result generated by the second thread bundle that performs the accumulation task, and the first thread bundle is also used to initialize the accumulation register in the target register.

10. The method according to claim 6, characterized in that, When the second thread bundle executing the accumulation task completes, the second thread bundle executing the storage task is used to write the intermediate results in the shared register to the system memory; wherein, the private register corresponding to the second thread bundle executing the storage task is used to store the third memory address of the target matrix block.

11. The method according to any one of claims 4 to 10, characterized in that, Once the task of the first thread bundle is completed, the private register corresponding to the first thread bundle is released.

12. The method according to any one of claims 4 to 10, characterized in that, The target register is a physical space of a preset size; the preset size is determined based on the first size of the shared register and the second size of the private register corresponding to each of the second thread bundles.

13. The method according to any one of claims 4 to 10, characterized in that, The second size of the private registers corresponding to each of the second thread bundles is the same; and / or, the second size of the private register corresponding to any of the second thread bundles is the same as the third size of the private register corresponding to the first thread bundle.

14. The method according to any one of claims 4 to 10, characterized in that, The logical address range of the first thread bundle is the sum of the logical address range of the common register and the logical address range of the private register corresponding to the first thread bundle; the logical address range of the second thread bundle is the sum of the logical address range of the common register and the logical address range of the private register corresponding to the second thread bundle.

15. The method according to claim 14, characterized in that, The method further includes: Based on the physical base address and logical address offset of the shared register, the target physical address corresponding to the logical address of the first thread beam request is determined.

16. The method according to claim 14, characterized in that, The target register is a contiguous physical space, and the method further includes: If the logical address of the second thread beam request is less than the maximum logical address of the shared register, the target physical address corresponding to the logical address of the second thread beam request is determined based on the physical base address and logical address offset of the shared register. If the logical address requested by the second thread bundle is greater than or equal to the maximum logical address of the shared register, the target physical address corresponding to the logical address requested by the second thread bundle is determined based on the physical base address of the shared register, the identifier of the thread bundle, the logical address range and address offset of the private register corresponding to the second thread bundle.

17. The method according to any one of claims 4 to 10, characterized in that, The method further includes: In response to the completion of all instruction execution by the first thread bundle, the shared register is retained.

18. The method according to any one of claims 4 to 10, characterized in that, The method further includes: In response to the completion of all instruction execution in each of the second thread bundles, the target register is released.

19. The method according to any one of claims 4 to 10, characterized in that, The generation priority of the at least two types of second thread bundles is higher than the generation priority of the first thread bundle, and the method further includes: In response to a new matrix multiplication task being assigned to the processor unit, and the first thread bundle of the matrix multiplication task completing all instruction execution, at least two types of second thread bundles for the matrix multiplication task are generated. After generating the at least two types of second thread bundles, a new first thread bundle is generated; the new first thread bundle is used at least to allocate a new target register for the new matrix multiplication task in the processor unit.

20. A matrix multiplication task execution device, characterized in that, A processor unit used in a graphics processor, the device comprising: An execution module is configured to allocate a target register for the matrix multiplication task in the processor unit during the execution of the matrix multiplication task in the processor unit, wherein the target register includes a shared register; A generation module is used to generate at least two types of second thread bundles for performing the matrix multiplication task; different types of second thread bundles are used to implement different functions; the at least two types of second thread bundles asynchronously execute the matrix multiplication task based on the shared register; the shared register is used to store the intermediate results of the matrix multiplication task; the shared register is visible to the at least two types of second thread bundles; the intermediate results include multiply-accumulate results, and the shared register is used to store the multiply-accumulate results.

21. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 19.

22. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 19.

23. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 19.

Images