Thread group concurrent execution control method and device, electronic equipment and storage medium

Abstract

The invention provides a thread group concurrent execution control method, which is applied to a computing unit in a processor, and comprises the following steps: when a first thread group and a second thread group are received in sequence, if the second thread group is received before the execution of the first thread group is completed, executing the first thread group; determining whether the computing unit meets a condition of executing two thread groups at the same time or not; and in response to determining that the computing unit meets the condition of simultaneously executing the two thread groups, starting to execute the head thread bundle of the second thread group after the execution of the head thread bundle of the first thread group is completed, and starting to execute the non-head thread bundle of the second thread group after the execution of the non-head thread bundle of the first thread group is completed. According to the method, the execution efficiency of the thread group on the single computing unit in the processor can be improved, and the execution time of the whole Tcore enable kernel is shortened.

Description

Technical Field

[0001] This application relates to the technical field of artificial intelligence chip design, and particularly relates to a method, device, electronic device and storage medium for controlling concurrent execution of thread groups. Background Art

[0002] Tcore (Tensor Core) is a dedicated computing unit in processors such as GPUs (Graphics Processing Units) and GPGPUs (General-Purpose Graphics Processing Units), mainly used to accelerate matrix operations. Especially in deep learning and high-performance computing, it improves computing efficiency by performing mixed-precision calculations (such as FP16 and FP32). High-performance computing kernels can be written using Tcore to execute specific matrix operations (such as matrix multiplication), and such kernels are also called Tcore enable kernels.

[0003] In related technologies, a Tcore enable kernel can contain multiple TGs (Thread Groups), and these TGs are allocated to different CUs (Compute Units) in the processor for parallel execution. When a thread group in a Tcore enable kernel is executed by a single CU in the processor, it occupies all the resources of that CU, such as the group shared memory (gsm) and thread group barrier (tg bar) at the thread group level. Therefore, a single CU can only execute one TG in the Tcore enable kernel at a time, and the CU can execute the next TG only after the current TG has finished execution, resulting in low execution efficiency of the TG on a single CU. Moreover, when the number of TGs in the Tcore enable kernel is more than the number of CUs in the GPU, two TGs will be serially executed on the same CU, resulting in a long execution time for the entire Tcore enable kernel. Summary of the Invention

[0004] In view of this, this application provides a method, device, electronic device and storage medium for controlling concurrent execution of thread groups, which can improve the execution efficiency of thread groups on a single computing unit (CU) in the processor and reduce the execution time of the entire Tcore enable kernel.

[0005] To solve the above technical problems, the technical solution of the present application is implemented as follows:

[0006] In one embodiment, a method for controlling concurrent execution of thread groups is provided, which is applied to a computing unit in a processor. The method includes:

[0007] When receiving a first thread group and a second thread group in sequence, if the second thread group is received before the first thread group finishes execution, determine whether the computing unit meets the condition for concurrently executing two thread groups;

[0008] In response to determining that the computing unit meets the condition for concurrently executing two thread groups, start executing the warp of the second thread group after the warp of the first thread group finishes execution, and start executing the non-warp of the second thread group after the non-warp of the first thread group finishes execution.

[0009] In a possible implementation, the method further includes: providing a configurable concurrency mode switch for allowing upper-layer software to select to enable or disable the multi-thread group concurrent execution function;

[0010] The determining whether the computing unit meets the condition for concurrently executing two thread groups includes:

[0011] If the concurrency mode switch is enabled and the computing unit has the first hardware resources required for concurrently executing two thread groups, determine that the computing unit meets the condition for concurrently executing two thread groups; otherwise, determine that the computing unit does not meet the condition for concurrently executing two thread groups.

[0012] In a possible implementation, the first hardware resources include: shared memory at the thread group level.

[0013] In a possible implementation, the starting to execute the warp of the second thread group after the warp of the first thread group finishes execution, and starting to execute the non-warp of the second thread group after the non-warp of the first thread group finishes execution includes:

[0014] Allocating execution units for each warp in the second thread group;

[0015] For each execution unit, after the warp of the first thread group allocated to this execution unit finishes execution, issue the warp of the second thread group allocated to this execution unit to this execution unit for execution, and after the non-warp of the first thread group allocated to this execution unit finishes execution, issue the non-warp of the second thread group allocated to this execution unit to this execution unit for execution.

[0016] Wherein, the first thread group and the second thread group are thread groups in a kernel enabling tensor cores.

[0017] In a possible implementation, the first thread group and the second thread group are thread groups in the same core.

[0018] In another possible implementation, the first thread group and the second thread group are thread groups in different cores of the same process. The first thread group and the second thread group have the same thread group cluster size, and the computing unit has second hardware resources required for parallel execution of two thread groups belonging to different cores.

[0019] In a possible implementation, the second hardware resources include at least one of the following hardware resources: constant registers, thread group barriers, thread group cluster-level shared memory barriers, and thread-level registers.

[0020] In another embodiment, a thread group concurrent execution control device is provided, which is applied to a computing unit in a processor. The device includes:

[0021] A condition determination unit, when receiving the first thread group and the second thread group in sequence, determines whether the computing unit meets the condition for simultaneously executing the two thread groups if the second thread group is received before the first thread group finishes execution;

[0022] An execution control unit, configured to, in response to determining that the computing unit meets the condition for simultaneously executing the two thread groups, start executing the warp of the second thread group after the warp of the first thread group finishes execution, and start executing the non-warp of the second thread group after the non-warp of the first thread group finishes execution.

[0023] In another embodiment, an electronic device is provided, including:

[0024] A processor;

[0025] A memory for storing executable instructions of the processor;

[0026] Wherein, the processor is configured to execute the executable instructions to implement the thread group concurrent execution control method as described in any one of the above.

[0027] In another embodiment, a computer-readable storage medium is provided. When at least one instruction in the computer-readable storage medium is executed by a processor of an electronic device, the electronic device can implement the thread group concurrent execution control method as described in any one of the above.

[0028] In another embodiment, a computer program product is provided, including a computer program, characterized in that when the computer program is executed by a processor, it implements the thread group concurrent execution control method as described in any one of the above.

[0029] As can be seen from the above technical solution, in the above embodiment, when the computing unit (CU) receives the second thread group before the first thread group finishes execution, if the computing unit meets the condition for executing two thread groups simultaneously, it can start executing the warp of the second thread group immediately after the warp of the first thread group finishes execution, rather than waiting for all warps of the first thread group to finish execution before starting to execute the warp of the second thread group. Thus, the parallel execution of two thread groups in the computing unit is achieved, which can improve the execution efficiency of the thread groups in a single computing unit and reduce the execution time of the entire Tcoreenable kernel. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0031] Figure 1 [[ID=,10]]Schematic diagram of the execution process of multiple TGs in a single CU in the related art;

[0032] Figure 2 Schematic diagram of the execution process of the kernel on the CU in the related art;

[0033] Figure 3 Schematic diagram of the architecture of the artificial intelligence chip provided by the embodiment of the present invention;

[0034] Figure 4 Schematic diagram of the flow of the thread group concurrent execution control method in the embodiment of the present application;

[0035] Figure 5 Schematic diagram of the flow for determining whether the CU meets the condition for executing two thread groups simultaneously in the embodiment of the present application;

[0036] Figure 6 Schematic diagram of the flow of concurrent execution of two TGs in the embodiment of the present application;

[0037] Figure 7 Schematic diagram of the concurrent execution process of two TGs in a single CU in the embodiment of the present application;

[0038] Figure 8 Schematic diagram of the execution process of the warps of two TGs in a single EU in the embodiment of the present application;

[0039] Figure 9 Schematic diagram of the parallel execution of multiple thread groups in a single CU in the embodiment of the present application;

[0040] Figure 10 This is a schematic structural diagram of the thread group concurrent execution control device according to an embodiment of the present application;

[0041] Figure 11 This is a schematic structural diagram of an electronic device according to an embodiment of the present application. Detailed implementation manners

[0042] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0043] In the specification and claims of the present invention and the above-mentioned accompanying drawings, terms such as "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects, and do not necessarily need to be used to describe the order or sequence of the target. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0044] In the related art, the TG in Tcore enable kernel is serially executed on a single CU of the processor. Specifically, as Figure 1 shown, after the first TG allocated to the CU is executed, the second TG allocated to the CU will start to be executed. After the second TG allocated to the CU is executed, the third TG allocated to the CU will start to be executed, and so on. This method of serially executing TG on a single CU has low execution efficiency.

[0045] Moreover, when the number of TGs in Tcore enable kernel is greater than the number of CUs in the GPU, there will be a situation where multiple TGs are allocated to the same CU. Specifically, as Figure 2 shown, the number of TGs in Tcore enable kernel is 4, and the number of CUs in the GPU is 3. Among them, CU1 and CU2 are each allocated one TG, while CU3 is allocated 2 TGs. Since these two TGs are serially executed, the execution time of the entire Tcore enable kernel is lengthened.

[0046] Based on the above technical problems, in the embodiments of the present application, a method for controlling concurrent execution of thread groups is disclosed. When the CU meets the condition of simultaneously executing two thread groups, during the execution of the first received thread group (e.g., TG0), the head warps of the later received thread group (e.g., TG1) can be executed in parallel, so as to achieve the concurrent execution of the two thread groups, thereby improving the execution efficiency of the TG in a single CU and reducing the execution time of the entire Tcore enable kernel.

[0047] The following will describe in detail the specific implementation of the method for controlling concurrent execution of thread groups in the embodiments of the present invention with reference to the accompanying drawings.

[0048] See Figure 3 , Figure 3 which is a schematic diagram of the architecture of an artificial intelligence chip provided by an embodiment of the present invention. The artificial intelligence chip can be a GPU or a GPGPU. As Figure 3 shown, the artificial intelligence chip internally includes multiple Streaming Processing Clusters (SPCs). Each SPC contains multiple computing units (the computing units can be CUs or SMs, hereinafter collectively referred to as CUs). Each CU contains multiple Execution Units (EUs). Among them, the multiple CUs in the SPC can form multiple CU clusters, and the CU clusters are used to run thread group clusters (i.e., TG clusters, composed of multiple TGs). The CU is the core component for executing computing tasks and is used to run TGs (a TG includes multiple warps, and the number of warps in a TG generally does not exceed 32). The EU is the smallest computing unit within the CU and is used to run the warps in the TG. The CU also includes other hardware resources, such as constant scalar registers (csr), thread group barriers (tg bar), cluster shared memory barriers (csmbar) at the thread group cluster level, group shared memory (gsm) at the thread group level, thread level registers (tlr), etc. Figure 3 Only some of the hardware resources are shown.

[0049] The method for controlling concurrent execution of thread groups provided by the embodiments of the present application can be applied to the above CU. For the specific implementation, please refer to Figure 4 .

[0050] See Figure 4 , Figure 4It is a schematic flowchart of the thread group concurrent execution control method in an embodiment of the present application. This method is applied to a computing unit (CU) in a processor, and the processor can be any one of a GPU and a GPGPU. For example, Figure 4 As shown, the method specifically includes the following steps:

[0051] Step 401, when receiving the first thread group (TG0) and the second thread group (TG1) successively, if TG1 is received before TG0 finishes execution, determine whether the CU meets the condition for simultaneously executing two thread groups.

[0052] In an embodiment of the present application, TG0 and TG1 are thread groups in Tcore enable kernel.

[0053] In an embodiment of the present application, if the CU receives TG1 before TG0 finishes execution, it determines whether to concurrently execute TG0 and TG1 by judging whether the CU meets the condition for simultaneously executing two thread groups. Among them, when the CU meets the condition for simultaneously executing two thread groups, TG0 and TG1 are concurrently executed, that is, step 302 is executed; when the CU does not meet the condition for simultaneously executing two thread groups, TG0 and TG1 are serially executed, that is, TG0 is executed first, and after TG0 finishes execution, TG1 is executed.

[0054] Step 402, in response to determining that the CU meets the condition for simultaneously executing two thread groups, start executing the head warp of TG1 after the head warp of TG0 finishes execution, and start executing the non-head warp of TG1 after the non-head warp of TG0 finishes execution.

[0055] In an embodiment of the present application, the head warp of a thread group is mainly used to execute copy instructions to achieve data transfer, for example, copying data from Figure 3 the hardware resource gsm (group shared memory) shown to Figure 3 the hardware resource glm (global memory) shown, copying data from glm to gsm, copying data from gsm to gsm, etc. The number of thread-level registers (tlr) usually used is relatively small. The non-head warp in a thread group refers to the warp other than the head warp in the thread group and is mainly used to execute core computing operations, such as matrix multiplication (MMA), convolution operation (conv), etc.

[0056] In the embodiments of the present application, after the head thread bundle of TG0 is executed, the head thread bundle of TG1 can be started immediately, without waiting for all the thread bundles of TG0 (including the head thread bundle and non-head thread bundles of TG0) to be completely executed (i.e., TG0 is executed), and then starting to execute TG1. Thus, the parallel execution of TG0 and TG1 is achieved, significantly improving the execution efficiency of TG in a single CU. Also, since the TGs assigned to the same CU can be executed concurrently, the time required for all the TGs in the entire kernel to be completely executed is significantly reduced.

[0057] In the embodiments of the present application, if the CU receives the third thread group TG2 before TG1 is completely executed, then TG1 and TG2 are regarded as two successively received thread groups, and the above steps 301 to 302 are executed to achieve the concurrent execution of TG1 and TG2.

[0058] In some embodiments, a configurable concurrency pattern switch can be provided to the upper-layer software, enabling the upper-layer software to select to enable or disable the multi-thread group concurrent execution function through this concurrency pattern switch.

[0059] In some embodiments, in step 401, to determine whether the CU meets the condition for simultaneously executing two thread groups, the specific implementation can be referred to Figure 5 。

[0060] Figure 5 FIG. is a schematic flowchart for determining whether the CU meets the condition for simultaneously executing two thread groups in the embodiments of the present application, and the specific steps are as follows:

[0061] Step 501: Determine whether the concurrency pattern switch is enabled;

[0062] Step 502: When the concurrency pattern switch is enabled, check the first hardware resources of the CU;

[0063] In some examples, the first hardware resources include shared memory at the thread group level, such as group shared memory (gsm).

[0064] Step 503: When it is determined through the check that the CU has the first hardware resources required for parallel execution of two thread groups, determine that the CU meets the condition for simultaneously executing two thread groups;

[0065] Step 504: When it is determined through the check that the CU does not have the first hardware resources required for parallel execution of two thread groups, determine that the CU does not meet the condition for simultaneously executing two thread groups.

[0066] For any TG in Tcore enable kernel, the number of header warps it contains is the same as the number of EUs included in the CU. After the CU receives the TG, it allocates EUs to each warp in the TG. Among them, each EU will be allocated a header warp of the TG.

[0067] Therefore, in the embodiment of the present application, after the CU receives TG0, it allocates EUs to each warp in TG0, so that each EU in the CU is allocated a header warp of TG0. Similarly, after the CU receives TG1, it allocates EUs to each warp in TG1, so that each EU in the CU is allocated a header warp of TG1.

[0068] In some embodiments, in step 402, after the header warp of TG0 finishes execution, the header warp of TG1 starts to execute, and after the non-header warp of TG0 finishes execution, the non-header warp of TG1 starts to execute. For specific implementation, refer to Figure 6 and Figure 7 。

[0069] Figure 6 This is a schematic diagram of the concurrent execution process of two TGs in the embodiment of the present application. The specific steps are as follows:

[0070] Step 601: Allocate EUs to each warp in TG1;

[0071] Step 602: For each EU, execute steps 603 to 604;

[0072] Step 603: After the header warp of TG0 allocated to this EU finishes execution, send the header warp of TG1 allocated to this EU to this EU for execution;

[0073] In the embodiment of the present application, the CU sends the header warp of TG1 to the EU to which this header warp is allocated, so that this EU executes this header warp of TG1.

[0074] Step 604: After the non-header warp of TG0 allocated to this EU finishes execution, send the non-header warp of TG1 allocated to this EU to this EU for execution.

[0075] In the embodiment of the present application, the CU sends the non-header warp of TG1 to the EU to which this non-header warp is allocated, so that this EU executes this non-header warp of TG1.

[0076] Step 605: The non-header threads of TG1 finish execution.

[0077] So far, the concurrent execution process of TG0 and TG1 ends. The concurrent execution process of TG0 and TG1 in a single CU is as Figure 7 shown.

[0078] When the CU meets the condition of simultaneously executing two thread groups, the CU allocates EUs to the warps in thread group TG0 and the warps in thread group TG1, so that each EU is allocated a head warp and some non-head warps of TG0, as well as a head warp and some non-head warps of TG1.

[0079] For any EU, after the EU finishes executing the head warp of TG0 allocated to the EU, the CU immediately issues the head warp of TG1 allocated to the EU to the EU for execution, so that the head warp of TG1 and the non-head warps of TG0 that have not been completed are executed in parallel; after the EU finishes executing the non-head warps of TG0 allocated to the EU, the CU immediately issues the non-head warps of TG1 allocated to the EU to the EU for execution.

[0080] The execution process of the warps of the two TGs in a single EU is described.

[0081] Figure 8 It is a schematic diagram of the execution process of the warps of multiple TGs in a single EU. As Figure 8 shown, the warps allocated to EU0 include: the head warp of TG0 (i.e., warp 0), warp 4, warp 8, and warp 12, as well as the head warp of TG1, warp 4, and warp 8, etc. Then the execution process of the warps in EU0 is as follows:

[0082] S81. EU0 first executes the head warp, warp 4, warp 8, and warp 12 of TG0 in parallel.

[0083] S82. After the head warp of TG0 is executed, the head warp of TG1 is issued to EU0. Then EU0 executes the head warp of TG1 in parallel, as well as the non-completed warps among warp 4, warp 8, and warp 12 of TG0.

[0084] S83. After the non-head warps of TG0 are executed, warp 4 and warp 8 of TG1 are issued to EU0. Then, EU0 executes warp 4 and warp 8 of TG1 in parallel.

[0085] In the embodiments of the present application, both TG0 and TG1 are thread groups in a Tcore-enabled kernel.

[0086] TG0 and TG1 can be thread groups in the same kernel, or can be thread groups in different kernels in the same process.

[0087] In some embodiments, TG0 and TG1 can be thread groups in the same kernel.

[0088] In some other embodiments, TG0 and TG1 may be thread groups in different kernels. When TG0 and TG1 belong to different kernels, these two kernels should belong to the same process. At the same time, the thread group cluster sizes corresponding to TG0 and TG1 are the same, and the CU has second hardware resources required for parallel execution of two thread groups belonging to different kernels. In one example, the second hardware resources include at least one of the following hardware resources: constant registers, thread group barriers, thread group cluster-level shared memory barriers, and thread-level registers.

[0089] The following Figure 9 is combined to illustrate the process of the CU parallelly executing multiple thread groups by way of example.

[0090] Figure 9 FIG. is a schematic diagram of parallel execution of multiple thread groups in a single CU in an embodiment of the present application. As Figure 9 shown, the CU successively receives thread groups TG0, TG1, and TG2. Among them, when the CU receives TG1, TG0 has not been executed completely yet, and when it receives TG2, TG1 has not been executed completely yet. The parallel execution process of TG0, TG1, and TG2 is as follows:

[0091] S91. Initialize and execute the warps in TG0 (including the head warp and non-head warps in TG0);

[0092] S92. When the head warp in TG0 finishes execution, initialize and execute the head warp in TG1. At this time, the head warp in TG1 and the warps in TG0 that have not been executed completely are executed in parallel;

[0093] S93. When the non-head thread groups in TG0 finish execution, initialize and execute the non-head warps in TG1;

[0094] S94. When the head warp in TG1 finishes execution, initialize and execute the head warp in TG2. At this time, the head warp in TG2 and the warps in TG1 that have not been executed completely are executed in parallel;

[0095] S95. When the non-head thread groups in TG1 finish execution, initialize and execute the non-head warps in TG2.

[0096] S96. The non-head warps in TG2.

[0097] So far, the parallel execution process of TG0, TG1, and TG2 ends.

[0098] As can be seen from the above embodiments, when TG0 has not completed execution, the head warp of TG1 has already started to execute in parallel, thereby advancing the execution time of TG1. When TG1 has not completed execution, the head warp of TG2 has already started to execute in parallel, thereby advancing the execution time of TG2. In this way, the total time required for the CU to execute TG0, TG1, and TG2 is shortened, thereby improving the TG execution efficiency of a single CU. Moreover, when the number of TGs included in a kernel is greater than the number of CUs in a GPU, due to the improvement of the TG execution efficiency of a single CU, the time for a single CU to complete the execution of multiple TGs assigned to this kernel is shortened. Therefore, the overall execution time of the entire kernel can be shortened. In addition, the improvement of the TG execution efficiency of TGs belonging to different kernels assigned to the same CU can also shorten the total execution time required for multiple kernels to execute in parallel.

[0099] Any combination of the above optional technical solutions can form optional embodiments of the present disclosure, which will not be elaborated one by one here.

[0100] Based on the same inventive concept, an embodiment of the present application further provides a thread group concurrent execution control device, which is applied to a computing unit in a processor, and the processor can be any one of a GPU and a GPGPU. Refer to Figure 10 , Figure 10 which is a schematic structural diagram of the thread group concurrent execution control device according to an embodiment of the present application. As Figure 10 shown, the device includes:

[0101] A condition determination unit 1001, when receiving a first thread group TG0 and a second thread group TG1 in sequence, if TG1 is received before TG0 completes execution, determines whether the computing unit meets the condition for simultaneously executing two thread groups;

[0102] An execution control unit 1002, configured to, in response to determining that the computing unit meets the condition for simultaneously executing two thread groups, start executing the head warp of TG1 after the head warp of TG0 completes execution, and start executing the non-head warp of TG1 after the non-head warp of TG0 completes execution.

[0103] In some embodiments,

[0104] The device further includes: a switch providing unit 1003, configured to provide a configurable concurrent mode switch, and the concurrent mode switch is used to allow upper-layer software to select to enable or disable the multi-thread group concurrent execution function;

[0105] The condition determination unit 1001 determines whether the computing unit meets the condition for simultaneously executing two thread groups, including:

[0106] If the concurrent mode switch is enabled and the computing unit has the first hardware resources required to execute two thread groups in parallel, it is determined that the computing unit meets the condition for executing the two thread groups simultaneously; otherwise, it is determined that the computing unit does not meet the condition for executing the two thread groups simultaneously.

[0107] In some embodiments,

[0108] The first hardware resource includes: shared memory at the thread group level.

[0109] In some embodiments,

[0110] The execution control unit 1002 starts executing the head warp of TG1 after the head warp of TG0 is executed, and starts executing the non-head warps of TG1 after the non-head warps of TG0 are executed, including:

[0111] Allocate execution units to each thread warp in TG1;

[0112] For each execution unit, after the execution of the head thread bundle of TG0 assigned to the execution unit is completed, the head thread bundle of TG1 assigned to the execution unit will be sent to the execution unit for execution, and after the execution of the non-head thread bundle of TG0 assigned to the execution unit is completed, the non-head thread bundle of TG1 assigned to the execution unit will be sent to the execution unit for execution.

[0113] In some embodiments,

[0114] The TG0 and the TG1 are thread groups in the kernel in which Tcore is enabled.

[0115] In some embodiments,

[0116] The TG0 and the TG1 are thread groups in the same kernel.

[0117] In other embodiments,

[0118] The TG0 and the TG1 are thread groups in different kernels of the same process, the TG0 and the TG1 have the same thread group cluster size, and the CU has second hardware resources required for parallel execution of the two thread groups belonging to different kernels.

[0119] In some embodiments,

[0120] The second hardware resource includes at least one of the following hardware resources: a constant register, a thread group barrier, a thread group cluster-level shared memory barrier, and a thread-level register.

[0121] The units in the above embodiments can be integrated into one body or deployed separately; they can be combined into one unit or further split into multiple sub-units.

[0122] Figure 11 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present application. In some embodiments, the electronic device is a test platform. The electronic device 1100 may vary greatly due to configuration or performance differences, and may include one or more processors (Central Processing Units, CPUs) 1101 and one or more memories 1102. Among them, at least one program code is stored in the memory 1102, and the at least one program code is loaded and executed by the processor 1101 to implement the methods for testing a processor provided in the above various embodiments. Of course, the electronic device 1100 may also have components such as a wired or wireless network interface, a keyboard, and an input / output interface for input and output. The electronic device 1100 may also include other components for implementing the functions of the device, which will not be elaborated here.

[0123] In an exemplary embodiment, there is also provided a computer-readable storage medium including at least one instruction, such as a memory including at least one instruction. The at least one instruction can be executed by a processor in a computer device to complete the method for testing a processor in the above embodiments.

[0124] Optionally, the above computer-readable storage medium may be a non-transitory computer-readable storage medium. For example, the non-transitory computer-readable storage medium may include a ROM (Read-Only Memory), a RAM (Random-Access Memory), a CD-ROM (Compact Disc Read-Only Memory), magnetic tape, a floppy disk, and an optical data storage device, etc.

[0125] In an exemplary embodiment, there is also provided a computer program product including a computer program, and when the computer program is executed by a processor, it implements the thread group concurrent execution control method provided in the above various embodiments.

[0126] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative labor.

[0127] The flowcharts and block diagrams in the accompanying drawings of the present application illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments disclosed in the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in the order marked in different drawings. For example, two consecutively represented blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as combinations of blocks in the block diagram or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0128] Those skilled in the art can understand that the features recited in the various embodiments and / or claims of the present application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly recited in the present application. In particular, without departing from the spirit and teachings of the present application, the features recited in the various embodiments and / or claims of the present application can be combined and / or combined in various ways, and all such combinations and / or combinations fall within the scope of the present application.

[0129] Specific embodiments are used herein to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention, and is not used to limit the present application. For those skilled in the art, changes can be made in the specific implementation manners and application scopes according to the ideas, spirits, and principles of the present invention. Any modifications, equivalent replacements, improvements, etc. made by them shall be included within the scope of protection of the present application.

Claims

1. A method for controlling concurrent execution of a thread group, applied to a computing unit in a processor, characterized in that The method includes: When a first thread group and a second thread group are received successively, if the second thread group is received before the first thread group finishes execution, determine whether the computing unit meets the condition for executing two thread groups simultaneously; In response to determining that the computing unit meets the condition for executing two thread groups simultaneously, start executing the warp of the second thread group after the warp of the first thread group finishes execution, and start executing the non-warp of the second thread group after the non-warp of the first thread group finishes execution.

2. The method according to claim 1, wherein: The method further includes: providing a configurable concurrency mode switch for allowing upper-layer software to select to enable or disable the multi-thread group concurrent execution function; The determining whether the computing unit meets the condition for executing two thread groups simultaneously includes: If the concurrency mode switch is enabled and the computing unit has the first hardware resources required for parallel execution of two thread groups, determine that the computing unit meets the condition for executing two thread groups simultaneously; otherwise, determine that the computing unit does not meet the condition for executing two thread groups simultaneously.

3. The method according to claim 2, characterized in that, The first hardware resources include: shared memory at the thread group level.

4. The method according to any one of claims 1 to 3, characterized in that, The starting to execute the warp of the second thread group after the warp of the first thread group finishes execution, and starting to execute the non-warp of the second thread group after the non-warp of the first thread group finishes execution includes: Allocating execution units for each warp in the second thread group; For each execution unit, after the warp of the first thread group allocated to this execution unit finishes execution, issue the warp of the second thread group allocated to this execution unit to this execution unit for execution, and after the non-warp of the first thread group allocated to this execution unit finishes execution, issue the non-warp of the second thread group allocated to this execution unit to this execution unit for execution.

5. The method according to any one of claims 1 to 3, characterized in that, The first thread group and the second thread group are thread groups in a kernel that enables tensor cores.

6. The method according to claim 5, characterized in that, The first thread group and the second thread group are thread groups in the same kernel.

7. The method according to claim 5, characterized in that The first thread group and the second thread group are thread groups in different kernels of the same process, the first thread group and the second thread group have the same thread group cluster size, and the computing unit has the second hardware resources required for parallel execution of two thread groups belonging to different kernels.

8. The method according to claim 7, characterized in that, The second hardware resources include at least one of the following hardware resources: constant registers, thread group barriers, thread group cluster-level shared memory barriers, thread-level registers.

9. A thread group concurrent execution control device, applied to a computing unit in a processor, characterized in that The apparatus includes: A condition determination unit, which when a first thread group and a second thread group are received successively, determines whether the computing unit meets the condition for executing two thread groups simultaneously if the second thread group is received before the first thread group finishes execution; An execution control unit, configured to start executing the warp of the second thread group after the warp of the first thread group finishes execution, and start executing the non-warp of the second thread group after the non-warp of the first thread group finishes execution in response to determining that the computing unit meets the condition for executing two thread groups simultaneously.

10. An electronic device, characterized in that, Includes: A processor; A memory for storing executable instructions of the processor; Wherein, the processor is configured to execute the executable instructions to implement the thread group concurrent execution control method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, When at least one instruction in the computer-readable storage medium is executed by a processor of an electronic device, the electronic device can implement the thread group concurrent execution control method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the thread group concurrent execution control method according to any one of claims 1 to 8.

Images