Method, computing device, medium, and program product for generating random launch of threads within a warp

Abstract

The present invention relates to a method, a computing device, a medium and a program product for generating random thread emission in a thread warp. The method comprises: configuring a thread local register in the thread warp to maintain a program counter of a subsequent expected execution of each thread in the thread warp, wherein the thread warp shares a program counter; in response to determining that the execution of a basic block of a program is completed, selecting an active thread as a representative thread; and using the program counter of the subsequent expected execution of the representative thread as the program counter of the subsequent expected execution of the thread warp. The present invention can not only avoid thread deadlock through a program solution, but also complete automatic transformation in a compiler.

Description

Technical Field

[0001] Embodiments of the present invention generally relate to the field of artificial intelligence, and more particularly to a method, a computing device, a computer-readable storage medium, and a computer program product for generating random launch of threads within a warp. Background Art

[0002] Single Instruction Multi Thread (SIMT) is a common instruction in GPU programming. For example, a warp consisting of 32 threads shares a program counter (PC). Each instruction will be executed by 32 threads and 32 thread local registers (TLRs) carrying their respective data. The graphics processing unit (GPU) provides a mask register to control whether each thread needs to be executed. The mask register is a 32-bit warp scalar register (WSR), in which each bit corresponds to the active state of a thread (for example, when the corresponding bit of the register is "1", it means that the corresponding thread is active and the corresponding instruction will be executed; when the corresponding bit of the register is "0", it means that the corresponding thread is inactive and the corresponding instruction will not be executed).

[0003] It should be understood that thread branching instructions are usually present in the intermediate representation ("Intermediate Representation", or "Intermediate Code", i.e., "intermediate code"). It should be understood that the intermediate representation is an intermediate data structure before the compiler generates the target code) to be executed and its basic blocks. In the traditional method for supporting thread branching, a mutual exclusion lock (mutex) shared by all threads is usually adopted. It should be understood that a mutual exclusion lock (also called a "thread lock") is a mechanism used in multi-threaded programming to prevent two threads from reading and writing the same common resource (such as a global variable) at the same time. When facing a loop instruction, the above-mentioned method of using a mutual exclusion lock shared by all threads is prone to cause thread deadlock. The so-called "thread deadlock" refers to a blocking phenomenon caused by competition for resources during the execution of two or more threads. If multiple threads are blocked at the same time, one or all of them are waiting for a resource to be released. The above-mentioned blocking may cause the program to fail to terminate normally. In the traditional program solution for avoiding thread deadlock, each thread warp is allowed to have only one thread to compete for resources, and then the threads in the thread warp are separated to execute the single-threaded part, thereby ensuring that only one thread in each thread warp enters the loop for obtaining resources. However, since there are many modes of mutex locks, the traditional program solution for avoiding thread deadlock cannot be automatically transformed in the compiler.

[0004] In summary, the disadvantage of the traditional method is that it is difficult to avoid thread deadlock through program solutions while allowing the program solutions to be automatically transformed in the compiler. Summary of the invention

[0005] The present invention provides a method, a computing device, a computer-readable storage medium and a computer program product for generating random thread emission in a thread warp, which can not only avoid thread deadlock through program solutions, but also complete automatic transformation in a compiler.

[0006] According to a first aspect of the present invention, a method for generating random thread launches in a warp is provided. The method comprises: configuring a thread local register in the warp to maintain a program counter of a subsequent expected execution of each thread in the warp, wherein the warps share a program counter; in response to determining that a basic block of a program is executed, selecting an active thread as a representative thread; and using the program counter of the subsequent expected execution of the representative thread as the program counter of the subsequent expected execution of the warp.

[0007] In some embodiments, the method for generating random thread transmission within a thread warp also includes: determining, via a compiler backend, whether a program basic block within a function contains instructions that require synchronization within the thread warp; in response to determining that a program basic block within the function contains instructions that require synchronization within the thread warp, inserting a first prologue instruction sequence into the beginning of the program basic block to be executed, so as to complete synchronization between threads within the thread warp through the first prologue instruction sequence.

[0008] In some embodiments, the method for generating random thread launch within a warp further includes: in response to determining that a basic block within the function does not include instructions requiring synchronization within the warp, inserting a second prologue instruction sequence into the beginning of the basic block to be executed.

[0009] In some embodiments, the method for generating random thread launch within a warp further includes: generating an end instruction sequence, so as to execute the generated end instruction sequence after the original instructions of the basic block of the program are executed and before the jump instruction is executed.

[0010] In some embodiments, the method for generating random thread transmission within a thread warp also includes: calculating a jump condition; updating a program counter of each thread's subsequent expected execution using a program counter identifier of a first jump target program basic block or a program counter identifier of a second jump target program basic block based on whether the jump condition is satisfied or not; using a program counter of a current representative thread's subsequent expected execution as a program counter of a subsequent expected execution of the thread warp; selecting a next active thread as a representative thread; and jumping to a program counter of a subsequent expected execution of the thread warp to start executing subsequent instructions.

[0011] In some embodiments, generating a second prologue instruction sequence for a program basic block includes: determining whether a program counter of a current thread is the same as a starting program counter of a program basic block to be executed; in response to determining that the program counter of the current thread is the same as a starting program counter of a program basic block to be executed, updating a mask; and executing instructions within the program basic block to be executed.

[0012] In some embodiments, generating a first prologue instruction sequence for a program basic block includes: determining whether there is a thread whose program counter expected to be executed subsequently is less than the program counter of the program basic block that needs to be synchronized; in response to determining that there is a thread whose program counter expected to be executed subsequently is less than the program counter of the program basic block that needs to be synchronized, executing a management instruction; in response to determining that there is no thread whose program counter expected to be executed subsequently is less than the program counter of the program basic block that needs to be synchronized, updating a mask according to whether the program counter expected to be executed subsequently of the thread is the same as the starting program counter of the program basic block that needs to be synchronized; and executing the original instructions of the program basic block that needs to be synchronized.

[0013] In some embodiments, executing management instructions includes: screening threads whose program counters expected to be subsequently executed are smaller than the program counters of program basic blocks that need to be synchronized so as to update a mask; selecting a representative thread based on the updated mask; using the program counter of the representative thread expected to be subsequently executed as the subsequent program counter of the thread warp; and jumping to the program counter of the thread warp expected to be subsequently executed to start executing subsequent instructions.

[0014] In some embodiments, selecting an active thread as the representative thread includes: selecting an active thread as the representative thread from active threads in the current area in a round-robin manner.

[0015] In some embodiments, configuring thread local registers within a warp for maintaining a program counter that each thread successor in the warp expects to execute includes: confirming, via a compiler front end and a middle end, whether a mutex lock pattern exists in an intermediate representation; and in response to confirming that a mutex lock pattern exists in the intermediate representation, configuring thread local registers within the warp for maintaining a program counter that each thread successor in the warp expects to execute.

[0016] According to a second aspect of the present invention, a computing device is also provided. The computing device comprises: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the computing device can perform the method of the first aspect of the present invention.

[0017] According to a third aspect of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a machine, the method of the first aspect of the present invention is executed.

[0018] According to a fourth aspect of the present invention, there is further provided a computer program product, comprising a computer program, wherein when the computer program is executed by a machine, the method of the first aspect of the present invention is performed.

[0019] The beneficial effects of the present invention are that it can not only avoid thread deadlock through program solutions, but also because the program counter of the subsequent expected execution of the thread bundle is the program counter of the subsequent expected execution of the selected active representative thread, it will not be affected by the mode of the mutex lock, so automatic conversion can be completed in the compiler.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other features, advantages and aspects of the embodiments of the present invention will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements.

[0022] Figure 1 A schematic diagram of a computing device implementing a method for generating random thread emission within a warp according to an embodiment of the present invention is schematically shown.

[0023] Figure 2 A flow chart of a method for generating random launch of threads within a warp according to an embodiment of the present invention is shown.

[0024] Figure 3 A flow chart of a method for generating an epilogue instruction sequence according to an embodiment of the present invention is shown.

[0025] Figure 4 A flow chart of a method for inserting a prologue instruction sequence into the beginning of a basic block of a program to be executed according to some embodiments of the present invention is shown.

[0026] Figure 5 A flow chart of a method for inserting a second prologue instruction sequence into the beginning of a basic block of a program to be executed according to some embodiments of the present invention is shown.

[0027] Figure 6 A flow chart of a method for inserting a first prologue instruction sequence into the beginning of a basic block of a program to be executed according to some embodiments of the present invention is shown.

[0028] Figure 7 A schematic diagram schematically shows an area in a program flow graph.

[0029] In the various drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0030] The preferred embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.

[0031] As used herein, the term "including" and its variations mean open inclusion, i.e., "including but not limited to". The term "based on" means "based at least in part on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. may refer to different or the same objects.

[0032] As described above, Figure 7 The schematic diagram of a region 700 in a program flow graph is schematically shown. The region 700 includes, for example: a first program basic block 710 (or "BB0"), a second program basic block 720 (or "BB1"), a third program basic block 730 (or "BB2"), and a fourth program basic block 740 (or "BB1"). Figure 7 As shown, the first 16 threads meet the predetermined condition (the predetermined condition is, for example, "Cond=tid<16", where tid indicates a thread identifier) ​​and therefore need to execute the second program basic block (or "BB1"), while the next 16 threads do not meet the predetermined condition ("Cond=tid<16") and therefore need to execute the third program basic block (or "BB2"), so the GPU will execute BB1 ​​and BB2 respectively, and when executing BB1, the mask is, for example, 0x00FF, that is, the first 16 threads are active; when executing BB2, the mask is, for example, 0xFF00, that is, the next 16 threads are active. It should be understood that for a GPU based on the SIMT model, 32 threads share the same program counter (or "PC"), so the above-mentioned type of thread branching needs to be supported by hardware or software. There are two traditional methods for supporting thread branching: one is a method that is fully supported by hardware implementation. The other is a method that is supported by software implementation. The advantages of hardware implementation support are: the program counter of each thread can be saved and updated in real time, and at any time, it can switch to other threads to execute the program counter of the corresponding thread and update the mask at this time. However, this method has high requirements for hardware, requiring the hardware to fully manage the program counter and mask, and also needs to bear the additional performance loss caused by recording and updating the program counter and mask. The software implementation support mainly includes two specific methods. The first software implementation method mainly includes: recording the mask when each program basic block (or "basic block", BB) is executed, and calculating the mask of the next basic block to be executed through bit operations. The second software implementation method mainly includes: saving the position of the next expected execution PC of each thread, and after jumping to the new program basic block, comparing the current position with the expected execution position to determine whether the thread is active in the current program basic block.

[0033] Research shows that in the above traditional methods for supporting thread branching, the problem of mutex locks needs to be solved. In the traditional methods for supporting thread branching, a mutex lock shared by all threads is usually used. It should be understood that a mutex lock is a mechanism used in multi-threaded programming to prevent two threads from reading and writing the same common resource (such as a global variable) at the same time. Its main function is to slice the code into critical sections one by one to ensure that the common resources are accessed in a single-threaded manner within the critical section.

[0034] Generally speaking, the mutex is a global variable. Each thread will try to acquire the mutex through an atomic instruction (eg, AtomicCAS). The following Table 1 schematically shows an exemplary pseudo code for acquiring the mutex through AtomicCAS.

[0035] Table 1

[0036]

[0037] It should be understood that the AtomicCAS(&mutex, 1, 0) instruction compares the variable pointed to by the first parameter (e.g., "mutex") with the second parameter (e.g., "1"), and assigns the value to the third parameter (e.g., "0") if they are equal. The return value is the original value in the variable pointed to by the first parameter (e.g., "mutex"). For example, the atomic instruction AtomicCAS is first used to compare whether the resource exists, for example, to confirm whether the mutex is "1". If it is "1", it indicates that it is not locked and can be obtained, then it is replaced with "0", "0" indicates that it is locked; and the original value "1" in the variable (e.g., "mutex") is returned, otherwise it returns "0". In the while loop, if the resource cannot be obtained, it will continue to execute in the loop, and the thread that obtains the resource will execute the subsequent single-threaded part, and finally the atomic instruction AtomicExch is used to release the resource and reset mutex to "1". It should be understood that the above AtomicExch(&mutex, 1) instruction assigns the variable pointed to by the first parameter (eg, “mutex”) to the second parameter (eg, “1”).

[0038] It should be understood that only the thread that obtains the resource can leave the loop. In thread branch processing, when thread branches are merged, they usually wait until both thread branches are executed before executing the subsequent branch. For example, Figure 7As shown, BB2 will wait for BB0 and BB1 to complete before starting to execute, so as to reduce the number of times BB2 is repeatedly executed, and also avoid some special instructions in BB2 from being split and executed. In the above method, when all threads want to leave the loop, the program counter corresponding to the thread bundle will leave the loop. This leads to the situation of mutual exclusion lock, where only one thread obtains resources and wants to leave the loop, but other threads cannot leave the loop because they do not obtain resources, which leads to the program counter of the final thread bundle still executing in the loop, and cannot execute the subsequent instructions to release resources, thus causing thread deadlock.

[0039] Research has shown that in order to avoid thread deadlock, a program solution is to modify the code. For example, each thread bundle can have only one thread competing for resources, and then separate the threads in the thread bundle to execute the single-threaded part, thereby ensuring that only one thread in each thread bundle enters the loop to obtain resources. However, due to the large number of mutex lock modes, the above program solution for avoiding thread deadlock cannot be automatically transformed in the compiler. The user needs to modify the program according to different mutex lock modes. Therefore, the above program solution is not a long-term solution.

[0040] In summary, the disadvantage of the traditional method is that it is difficult to avoid thread deadlock through program solutions while allowing the program solutions to be automatically transformed in the compiler.

[0041] In order to at least partially solve one or more of the above problems and other potential problems, an exemplary embodiment of the present invention proposes a scheme for generating random thread emission within a warp. In the scheme, thread local registers are configured within the warp to maintain the program counter of the subsequent expected execution of each thread in the warp; an active thread is selected as a representative thread when the execution of a program basic block is completed; and the program counter of the subsequent expected execution of the representative thread is used as the program counter of the subsequent expected execution of the warp; the present invention can select an active thread as a representative thread to obtain resources when the current program basic block is executed and the next program basic block needs to be executed, so that random thread emission within the warp can be implemented in a software manner, so that the program counter (WPC) of the subsequent expected execution of the warp is related to the program counter (LTN.TPC) of the subsequent expected execution of the selected representative thread, without relying on all threads in the warp leaving the loop, so that the representative thread and the warp can obtain resources without being affected by the loop and mutex mode. Therefore, the present invention can not only avoid thread deadlock through program solutions, but also because the program counter of the subsequent expected execution of the thread warp is the program counter of the subsequent expected execution of the selected active representative thread, it will not be affected by the mutex mode, so automatic conversion can be completed in the compiler.

[0042] In addition, in the traditional method of avoiding thread deadlock, it is necessary to rely on the topological order, that is, the program basic block of the dependent item can be executed only after the program basic block of the dependent item is executed, that is, the thread with the uppermost topological order is given priority. In the method of the present invention, since the program counter representing the subsequent expected execution of the thread selected when the previous program basic block is executed is used as the program counter of the subsequent expected execution of the thread warp, further, the present invention does not need to rely on the topological order, that is, the thread of the previous dependent program basic block can obtain the resources released by the subsequent dependent program basic block.

[0043] Figure 1 A schematic diagram of a computing device 100 for implementing a method for generating random thread emission within a warp according to an embodiment of the present invention is shown schematically. Figure 1As shown, the computing device 100 may have one or more processing units, including special processing units such as a graphics processing unit (GPU), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or a general-purpose computing on graphics processing units (GPGPU), and a general-purpose processing unit such as a CPU. The computing device 100 also includes at least: a thread local register configuration unit 102, a representative thread selection unit 104, and a next program counter determination unit 106 for a thread warp. It should be understood that the above-mentioned thread local register configuration unit 102, the representative thread selection unit 104, and the next program counter determination unit 106 for a thread warp can be software modules, which, for example, run on one or more processing units configured by the computing device 100.

[0044] The thread local register configuration unit 102 is used to configure the thread local registers in the warp to maintain the program counter of each thread in the warp that is expected to be executed later. The warps share a program counter.

[0045] The representative thread selection unit 104 is configured to determine whether the execution of the basic block of the program is completed; and in response to determining that the execution of the basic block of the program is completed, select an active thread as a representative thread.

[0046] The next program counter determination unit 106 of the warp is configured to use the program counter representing the subsequent expected execution of the thread as the program counter of the subsequent expected execution of the warp.

[0047] The following will be combined Figure 2 A method 200 for generating random launches of threads within a warp according to an embodiment of the present invention is described. Figure 2 FIG. 2 is a flowchart of a method 200 for generating random thread launches within a warp according to an embodiment of the present invention. It should be understood that the method 200 may be implemented, for example, in Figure 1 The method 200 is executed at the described computing device 100. The method 200 may also include additional actions not shown and / or may omit the actions shown, and the scope of the present invention is not limited in this respect.

[0048] At step 202 , the computing device 100 configures thread local registers within a warp for maintaining a program counter of each thread in the warp that is subsequently expected to execute, the warps sharing one program counter.

[0049] Regarding the thread bundle (warp), it is the smallest unit of GPU program execution. Threads in the same warp execute the same instructions, that is, SIMT. Each warp shares a program counter (PC).

[0050] The program counter (PC) indicates the location of the instruction executed by the program. Specifically, it is used to indicate the address of the first instruction of the basic block of the program.

[0051] Regarding the thread local register (TLR), it is a register unique to each thread. After being configured, it is used to maintain the program counter of each thread in the thread warp that is expected to be executed later.

[0052] In some embodiments, a method for configuring a thread local register in a thread warp includes, for example: confirming whether there is a mutex mode in the intermediate representation via the compiler front end and the middle end; and in response to confirming that there is a mutex mode in the intermediate representation, configuring a thread local register in the corresponding thread warp for maintaining the program counter that each thread in the thread warp is expected to execute later. For example, in some embodiments, if there is an atomic instruction in the intermediate representation, and the data result of the atomic instruction is used as a judgment condition for loop jump, there may be a mutex mode (for example, there is a corresponding identifier to indicate the mutex mode). If it is confirmed via the compiler front end and the middle end that there is a mutex mode in the intermediate representation, step 202 is executed. In some embodiments, the relevant identifier for indicating the mutex mode can also be manually marked.

[0053] At step 204 , the computing device 100 determines whether the basic block of the program has been executed to completion. In some embodiments, if the computing device 100 determines that the basic block of the program has not been executed to completion, it waits at step 204 .

[0054] Regarding the basic block of a program, or "basic block" (BB for short), it is a linear (or "sequentially executed") sequence of instructions. A basic block of a program has only one entry and exit, and each operation in the basic block is executed in order without any forks. The entry of a basic block of a program is its first statement, which can be transferred by a conditional transfer statement or an unconditional transfer statement. The exit of a basic block of a program is its last statement. It should be understood that the program flow graph uses basic blocks of a program as nodes. For example, Figure 7A first program basic block BB0, a second program basic block BB1, a third program basic block BB2 and a fourth program basic block BB3 are indicated, wherein BB1 or BB2 is a predecessor program basic block of BB3, and BB3 is a successor program basic block of BB1 or BB2.

[0055] In some embodiments, the computing device 100 determines whether the current basic block (for example but not limited to BB1) is executed. In some embodiments, the computing device 100 determines whether the basic block is executed for each basic block in the current block.

[0056] At step 206 , if the computing device 100 determines that the execution of the program basic block is complete, an active thread is selected as a representative thread.

[0057] For example, if the computing device 100 determines that the current program basic block has been executed and needs to execute the next program basic block, an active thread is selected as a representative thread from the active threads in the current area based on a polling scheduling method.

[0058] Regarding round-robin scheduling, it usually refers to allocating time slices of equal length to each process in sequence in a constant order. In the present invention, based on round-robin scheduling, each thread is made a leading thread in a sequential and cyclic manner.

[0059] Regarding a region, it is a combination of program basic blocks, and the combination of program basic blocks only includes one program basic block as an entry and one program basic block as an exit. It should be understood that a region can correspond to a function or a part of a function. For example Figure 7 A region 700 is indicated. The region 700 includes a first program basic block 710 (i.e., BB0), a second program basic block 720 (i.e., BB1), a third program basic block 730 (i.e., BB2), and a fourth program basic block 740 (i.e., BB3). BB0 is the only entrance to the region 700. BB3 is the only exit from the region 700.

[0060] For example, the jump condition of BB0 is "Cond = tid < 16", so the program basic block that the first 16 threads (the thread identifications of the first 16 threads are, for example, "0" to "15") expect to execute is, for example, BB0->BB1->BB3, and the program basic block that the second 16 threads (the thread identifications of the second 16 threads are, for example, "16" to "31") expect to execute is, for example, BB0->BB2->BB3.

[0061] Regarding the leading thread, for example, it is an active thread selected from a plurality of active threads in the current region.

[0062] Regarding active threads, they are, for example, threads that meet the execution conditions (or are "selected") and have the resources (e.g., registers and shared memory) required for execution in place (e.g., resources have been allocated to the threads). For example, for BB1, threads with thread identifiers "0" to "15" are active threads. For BB2, threads with thread identifiers "15" to "31" are active threads.

[0063] At step 208 , the computing device 100 uses the program counter representing the subsequent expected execution of the thread as the program counter of the subsequent expected execution of the warp.

[0064] For example, the program counter representing the subsequent expected execution of the thread is, for example, LTN .TPC. The computing device 100 makes the program counter (WPC) of the subsequent expected execution of the warp the same as LTN .TPC.

[0065] In the above scheme, thread local registers are configured in the thread bundle to maintain the program counter of the subsequent expected execution of each thread in the thread bundle; an active thread is selected as a representative thread when the program basic block is executed; and the program counter of the subsequent expected execution of the representative thread is used as the program counter of the subsequent expected execution of the thread bundle; the present invention can select an active thread in the current area as a representative thread to obtain resources when the current program basic block is executed and the next program basic block needs to be executed. Therefore, the present invention can realize random thread emission in the thread bundle (warp) by software, so that the program counter (WPC) of the subsequent expected execution of the warp is related to the program counter (LTN.TPC) of the subsequent expected execution of the selected representative thread, and does not depend on all threads in the thread bundle leaving the loop, so that the representative thread and the warp can obtain resources without being affected by the loop and mutex mode. Therefore, the present invention can not only avoid thread deadlock through program solutions, but also because the program counter of the subsequent expected execution of the thread bundle is the program counter of the subsequent expected execution of the selected active representative thread, it will not be affected by the mutex mode, so it can be automatically transformed in the compiler.

[0066] In some embodiments, the method 200 further includes: the computing device 100 topologically sorts all program basic blocks according to the program flow graph; and makes the program counter of the predecessor of each current program basic block smaller than the program counter of the current program basic block except for the loop back edge. By adopting the above means, the present invention can arrange the program basic blocks in a topological order, so as to facilitate the subsequent generation of a pre-order instruction sequence and an end instruction sequence for the program basic blocks in the function.

[0067] In some embodiments, method 200 further includes a method 300 for generating an ending instruction sequence. For example, computing device 100 generates an ending instruction sequence for executing the generated ending instruction sequence after the original instructions of the basic block of the program are executed and before the jump instruction is executed.

[0068] The following will be combined Figure 3 A method 300 for generating an epilogue instruction sequence according to an embodiment of the present invention is described. Figure 3 FIG. 3 is a flow chart of a method 300 for generating an end instruction sequence according to an embodiment of the present invention. It should be understood that the method 300 may be implemented in Figure 1 The method 300 is executed at the described computing device 100. The method 300 may also include additional actions not shown and / or may omit the actions shown, and the scope of the present invention is not limited in this respect.

[0069] It should be understood that the method 300 needs to be executed after the original instructions are executed at the end of the basic block of the program and before the jump instruction is executed.

[0070] At step 302 , the computing device 100 calculates a jump condition.

[0071] For example, as shown in Table 2, the jump condition is "Cond = tid < 16" (ie, the jump condition is: the thread ID is less than 16). For example, the computing device 100 calculates whether the jump condition "Cond = tid < 16" is satisfied according to the current thread ID.

[0072] At step 304, the computing device 100 updates the program counter of each thread that is expected to execute subsequently using the program counter identifier of the first jump target program basic block or the program counter identifier of the second jump target program basic block based on whether the jump condition is satisfied.

[0073] Regarding the program counter, it is a register in the processor (such as a GPU) that contains the address (location) of the instruction currently being executed. As each instruction is fetched, the storage address of the program counter is incremented by one. After each instruction is fetched, the program counter points to the next instruction in the sequence.

[0074] by Figure 7 For example, the second program basic block BB1 is the first jump target program basic block; and the third program basic block BB2 is the second jump target program basic block. For example, as indicated by the instruction "TPC = Cond? BB1_PC: BB2_PC" in Table 2, the computing device 100 uses the program counter identifier of BB1 (e.g., BB1_PC, which indicates the program counter identifier of the first jump target program basic block of BB1) or the program counter identifier of BB2 (e.g., BB2_PC, which indicates the program counter identifier of the second jump target program basic block) to update the program counter (TPC) of the thread that is expected to be executed later according to the state of whether the jump condition "Cond = tid < 16" is satisfied or not. For example, BB1_PC or BB2 is used as an operand to update the location of the TPC of each thread. For example, if the current thread identifier (tid) is 10, which is less than 16, BB1_PC is used to update the TPC. For example, if the current thread identifier (tid) is 20, which is greater than 16, BB2_PC is used to update the TPC.

[0075] At step 306 , the computing device 100 uses the program counter currently representing the subsequent expected execution of the thread as the program counter of the subsequent expected execution of the warp.

[0076] That is, the WPC of the thread warp is updated using the program counter (LTN.TPC) of the subsequent expected execution of the current representative thread. For example, as indicated by the instruction "WPC = LTN.TPC", the program counter (i.e., LTN.TPC) of the subsequent expected execution of the current representative thread (the representative thread identifier is LTN) is used as the subsequent program counter (i.e., WPC) of the thread warp.

[0077] Regarding the current representative thread, it is an active thread in a selected current region.

[0078] At step 308, the computing device 100 selects the next active thread as the representative thread. It should be understood that by selecting the next active thread as the representative thread, thread switching is achieved.

[0079] For example, the computing device 100 randomly selects the next active thread as the representative thread, for example, based on a round-robin scheduling (Round-Robin:). It should be understood that the next active thread should be selected from the active threads in the current region. For example, as indicated by the instruction "LTN = Round-Robin (LTN)" in Table 2.

[0080] At step 310 , the computing device 100 jumps to the program counter of the subsequent desired execution of the warp to start executing subsequent instructions.

[0081] For example, as indicated by the instruction “JUMP WPC” in Table 2, the computing device 100 jumps to the location of the WPC of the warp to start executing subsequent instructions.

[0082] The following Table 2 schematically shows an exemplary pseudo code of the epilogue instruction sequence.

[0083] Table 2

[0084]

[0085] By adopting the above means, the present invention can not only realize the generation of random thread emission within a thread warp, but also complete the switch instruction at the end of the program basic block.

[0086] In some embodiments, the method 200 further includes: a method for inserting a prologue instruction sequence into the beginning of a basic block of a program to be executed. Figure 4 FIG. 4 is a flowchart of a method 400 for inserting a preamble instruction sequence into the beginning of a basic block of a program to be executed according to some embodiments of the present invention. It should be understood that the method 400 may be implemented, for example, in Figure 1 The method 400 is executed at the described computing device 100. The method 400 may also include additional actions not shown and / or may omit the actions shown, and the scope of the present invention is not limited in this respect.

[0087] At step 402 , the computing device 100 determines, via the compiler back end, whether a basic block of a program within a function includes instructions that require intra-warp synchronization.

[0088] For example, if the compiler backend determines that a program basic block in a function includes at least one of the following instructions: a shuffle instruction, a synchronization instruction, and a return instruction, then it is determined that the program basic block in the function contains instructions that require intra-warp synchronization.

[0089] At step 404, if the computing device 100 determines that the program basic block within the function contains instructions that require synchronization within the thread warp, the first prologue instruction sequence is inserted into the beginning of the program basic block to be executed to complete the synchronization between threads within the thread warp through the first prologue instruction sequence.

[0090] Regarding the method for inserting the first preamble instruction sequence into the beginning of the basic block of the program to be executed, it for example includes: the computing device 100 determines whether there is a thread whose subsequent expected execution program counter is smaller than the program counter of the basic block of the program to be synchronized; in response to determining that there is a thread whose subsequent expected execution program counter is smaller than the program counter of the basic block of the program to be synchronized, executes the management instruction; in response to determining that there is no thread whose subsequent expected execution program counter is smaller than the program counter of the basic block of the program to be synchronized, updates the mask according to whether the subsequent expected execution program counter of the thread is the same as the starting program counter of the basic block of the program to be synchronized; and executes the original instructions of the basic block of the program to be synchronized. The following will be combined with Figure 6 The method 600 for inserting the first prologue instruction sequence into the beginning of the basic block of the program to be executed is described in detail, which will not be repeated here.

[0091] At step 406 , if the computing device 100 determines that the program basic block within the function does not include instructions that require intra-warp synchronization, the second prologue instruction sequence is inserted into the beginning of the program basic block to be executed.

[0092] Regarding the method for inserting the second prologue instruction sequence into the beginning of the basic block of the program to be executed, for example, it includes: the computing device 100 determines whether the program counter of the current thread is the same as the starting program counter of the basic block of the program to be executed; in response to determining that the program counter of the current thread is the same as the starting program counter of the basic block of the program to be executed, updating the mask; and executing the instructions in the basic block of the program to be executed. Figure 5 The method 500 for inserting the second prologue instruction sequence into the beginning of the basic block of the program to be executed is described in detail, which will not be repeated here.

[0093] By adopting the above means, the present invention can generate different preamble instruction sequences based on whether the instructions of the program basic block in the function need intra-warp synchronization, so as to be inserted into the beginning of the program basic block.

[0094] The following combination Figure 5 and Figure 7 A method 500 for inserting a second prologue instruction sequence into the beginning of a basic block of a program to be executed is described. Figure 5 FIG. 5 is a flowchart of a method 500 for inserting a second prologue instruction sequence into the beginning of a basic block of a program to be executed according to some embodiments of the present invention. It should be understood that the method 500 may be implemented, for example, in Figure 1 The method 500 is executed at the described computing device 100. The method 500 may also include additional actions not shown and / or may omit the actions shown, and the scope of the present invention is not limited in this respect.

[0095] At step 502, the computing device 100 determines whether the program counter of the current thread is the same as the starting program counter of the basic block of the program to be executed.

[0096] Regarding the basic block of the program to be executed, it is, for example, BB1. Figure 7 As shown, BB1 includes two instructions, namely: “c = a + b” and “Jump BB3”.

[0097] At step 504, if the computing device 100 determines that the program counter of the current thread is the same as the starting program counter of the basic block of the program to be executed, the mask is updated.

[0098] For example, if the computing device 100 determines that the TPC of the current thread is the same as BB1_PC, the Mask is updated. For example, the Mask is set to "1" (eg, "1" indicates "true"), as indicated by the instruction "Mask = TPC == BB1_PC" in Table 3.

[0099] At step 506, the computing device 100 executes the instructions within the basic block of the program to be executed.

[0100] For example, computing device 100 executes instructions in BB1, such as executing "c = a + b".

[0101] By adopting various means of the above method 500, the present invention can be implemented by inserting the second prologue instruction sequence into the beginning of a basic block of the program.

[0102] It should be understood that after executing the instruction "c = a + b" in BB1, the following ending instruction sequence needs to be executed before executing the jump instruction "Jump BB3".

[0103] First, the computing device 100 uses the program counter identifier of the jump target program basic block to update the program counter of each thread that is subsequently expected to execute.

[0104] Regarding the jump target program basic block, it is, for example, Figure 7 The fourth program basic block (BB3) is shown.

[0105] For example, as indicated by the instruction “TPC = BB3_PC” in Table 3, the computing device 100 updates the program counter (TPC) of each thread that is subsequently expected to execute using the program counter identifier (BB3_PC) of the jump target program basic block (which is, for example, BB3).

[0106] The computing device 100 then uses the program counter currently representing the subsequent expected execution of the thread as the subsequent program counter of the warp.

[0107] For example, as indicated by the instruction “WPC = LTN.TPC” in Table 3, the computing device 100 uses the program counter (LTN.TPC) of the current representative thread's subsequent expected execution as the subsequent program counter (WPC) of the warp.

[0108] Furthermore, the computing device 100 selects the next active thread as the representative thread.

[0109] For example, as indicated by the instruction “LTN = Round-Robin (LTN)” in Table 3, the computing device 100 selects the next active thread as the representative thread based on a round-robin scheduling (Round-Robin:) method.

[0110] Afterwards, the computing device 100 jumps to the program counter of the subsequent desired execution of the warp to start executing the subsequent instructions, for example, as indicated by the instruction “JUMP WPC”.

[0111] Table 3 below schematically shows an exemplary pseudo code for inserting the second prologue instruction sequence into the beginning of the program basic block BB1 and executing the generated ending instruction sequence after the original instructions of the program basic block BB1 are executed and before the jump instruction is executed.

[0112] Table 3

[0113]

[0114] It should be understood that there are some warp-related instructions in the GPU program, such as data exchange instructions (shuffle, which allows threads to access each other's registers) and synchronization instructions (sync) between threads within the warp, and instructions that require synchronization within the warp. For these instructions that require synchronization within the warp, if such instructions are executed multiple times, it may cause semantic changes and may even cause thread deadlock. For basic blocks with these instructions, it is necessary to insert a synchronization-related preamble instruction sequence (for example, a first preamble instruction sequence) to the beginning of the basic block of the program to be executed, so as to complete the synchronization between threads within the warp through the first preamble instruction sequence.

[0115] The following combination Figure 6 and Figure 7 A method 600 is described for inserting a first prologue instruction sequence into the beginning of a basic block of a program to be executed. Figure 6 FIG. 6 is a flowchart of a method 600 for inserting a first prologue instruction sequence into the beginning of a basic block of a program to be executed according to some embodiments of the present invention. It should be understood that the method 600 may be implemented, for example, in Figure 1The method 600 is executed at the described computing device 100. The method 600 may also include additional actions not shown and / or may omit the actions shown, and the scope of the present invention is not limited in this respect.

[0116] At step 602 , the computing device 100 determines whether there is a thread whose program counter is expected to be executed later and is smaller than the program counter of the basic block of the program that needs to be synchronized.

[0117] The basic blocks of the program that need to be synchronized are, for example but not limited to: Figure 7 In some embodiments, BB3 includes, for example, a "Return c" instruction. In some embodiments, when the "Return c" instruction is called, it is required that the number of threads at the beginning of the execution of the function is such that all of the threads are active threads at the end of the execution of the function. That is, the threads involved in the execution of the function are made to "Return" together.

[0118] For example, the computing device 100 determines whether there is a program counter (eg, BB3_PC) whose TPC is less than a program basic block (eg, BB3) that needs to be synchronized, as indicated by the instruction “Cond = TPC < BB3_PC” in Table 4.

[0119] At step 604, if the computing device 100 determines that there is a program counter of a subsequent expected execution of a thread that is less than the program counter of a basic block of the program that requires synchronization, a management instruction is executed.

[0120] For example, as indicated by the instruction "Branch Cond, Handle, BB3_PC_new" in Table 4, if the jump condition Cond is met, the management instruction (e.g., Handle instruction) is executed; if the jump condition Cond is not met, the instruction (e.g., BB3_PC_new) for updating the program counter of the program basic block that needs to be synchronized is executed.

[0121] The execution of the management instruction includes, for example, steps 606 to 612 .

[0122] At step 606 , the computing device 100 screens threads whose program counters are expected to be executed subsequently are smaller than the program counters of the basic blocks of the program that need to be synchronized, so as to update the mask.

[0123] For example, as indicated by the instruction “Mask = TPC < BB3_PC” in Table 4, the computing device 100 filters the threads with TPC < BB3_PC so as to update the Mask based on the threads with TPC < BB3_PC.

[0124] At step 608 , the computing device 100 selects a representative thread based on the updated mask.

[0125] For example, the computing device 100 selects a representative thread from threads that have not been executed to BB3 based on a round-robin scheduling method, as indicated by the instruction “LTN = Round-Robin (LTN) (Mask)” in Table 4.

[0126] At step 610 , the computing device 100 uses the program counter representing the subsequent expected execution of the thread as the subsequent program counter of the warp.

[0127] For example, the computing device 100 uses LTN.TPC as the WPC of the Wrap, as indicated by the instruction “WPC = LTN.TPC” in Table 4.

[0128] At step 612 , the computing device 100 jumps to the program counter of the subsequent desired execution of the warp to start executing the subsequent instructions, as indicated by the instruction “Jump WPC” in Table 4.

[0129] At step 614, if the computing device 100 determines that there is no program counter of the subsequent expected execution of the thread that is smaller than the program counter of the program basic block that needs to be synchronized, the mask is updated according to the state of whether the program counter of the subsequent expected execution of the thread is the same as the starting program counter of the program basic block that needs to be synchronized.

[0130] For example, if the computing device 100 determines that there is no thread whose TPC is less than BB3_PC (i.e., all threads have reached BB3), the mask is updated according to whether TPC is the same as BB3_PC, as indicated by the instruction “Mask = TPC == BB3_PC” in Table 4.

[0131] At step 616, the computing device 100 executes the original instructions of the basic block of the program that requires synchronization.

[0132] For example, as indicated by the instruction "Return c" in Table 4, the computing device 100 exits BB3 and returns directly without updating TPC.

[0133] Table 4 below schematically shows an exemplary pseudo code for completing synchronization between threads in a warp by using the first prologue instruction sequence when the first prologue instruction sequence is inserted into the beginning of a basic block of a program to be executed.

[0134] Table 4

[0135]

[0136] In the above solution, the present invention implements synchronization between threads in a warp by inserting a first prologue instruction sequence into the beginning of the basic block of the program to be executed for a program basic block that requires synchronization between threads in the warp.

[0137] The various processes and processing described above, such as methods 200 to 500, may be performed at a computing device. The computing device, for example, includes: at least one processor (at least one graphics processor and at least one central processing unit); and a memory connected to the at least one processor in communication; wherein the memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor. In some embodiments, methods 200 to 500 may be implemented as a computer software program or program product, which is tangibly contained in a machine-readable medium. In some embodiments, part or all of the computer program may be loaded and / or installed on a computing device via a read-only memory (ROM) and / or a communication unit. When the computer program is loaded into a random access memory (RAM) and executed by a GPU and a CPU, one or more actions of methods 200 to 500 described above may be performed.

[0138] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions for executing various aspects of the present invention. The computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof.

[0139] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. Various aspects of the present invention are described herein with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each box of the flowchart and / or block diagram, and the combination of boxes in the flowchart and / or block diagram, can be implemented by computer-readable program instructions.

[0140] These computer-readable program instructions can be provided to a central processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the central processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0141] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the function involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0142] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0143] The above specific implementations do not constitute a limitation on the protection scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made according to design requirements and other factors.

Claims

1. A method for generating random launches of threads within a warp, characterized in that include: configuring thread local registers within the warp for maintaining a program counter of each thread in the warp that is expected to be executed subsequently, the warps sharing a program counter; In response to determining that execution of the basic block of the program is complete, selecting an active thread as a representative thread; as well as Using the program counter representing the subsequent expected execution of the thread as the program counter of the subsequent expected execution of the thread warp; Wherein, the method further comprises: Determine, via the compiler backend, whether a basic block of a program in the function contains instructions that require intra-warp synchronization; In response to determining that the program basic block in the function contains instructions that require intra-warp synchronization, a first prologue instruction sequence is inserted into the beginning of the program basic block to be executed, so as to complete synchronization between threads in the warp through the first prologue instruction sequence.

2. The method according to claim 1, characterized in that Also includes: In response to determining that the program basic block in the function does not include instructions requiring intra-warp synchronization, a second prologue instruction sequence is inserted into the beginning of the program basic block to be executed.

3. The method according to claim 1, characterized in that Also includes: An end instruction sequence is generated, so as to be executed after the original instructions of the basic block of the program are executed and before the jump instruction is executed.

4. The method according to claim 3, characterized in that Generate the final instruction sequence including: Calculate jump conditions; Based on whether the jump condition is satisfied or not, using the program counter identifier of the first jump target program basic block or the program counter identifier of the second jump target program basic block to update the program counter of each thread that is expected to be executed subsequently; Using the program counter of the subsequent expected execution of the current representative thread as the program counter of the subsequent expected execution of the warp; Selecting the next active thread to serve as the representative thread; and Jumps to the program counter of the subsequent desired execution of the warp to start executing the subsequent instructions.

5. The method according to claim 2, characterized in that: Inserting the second prologue instruction sequence into the beginning of the basic block of the program to be executed includes: Determine whether the program counter of the current thread is the same as the starting program counter of the basic block of the program to be executed; In response to determining that the program counter of the current thread is the same as the starting program counter of the basic block of the program to be executed, updating the mask; and Execute the instructions in the basic block of the program to be executed.

6. The method according to claim 1, characterized in that Inserting the first prologue instruction sequence into the beginning of the basic block of the program to be executed includes: determining whether there is a thread whose program counter is expected to be executed later is smaller than the program counter of the basic block of the program that needs to be synchronized; In response to determining that there is a thread whose subsequent expected execution program counter is less than the program counter of the program basic block that needs to be synchronized, executing the management instruction; in response to determining that there is no thread whose subsequent expected execution program counter is less than the program counter of the program basic block that needs to be synchronized, updating the mask according to whether the program counter of the subsequent expected execution of the thread is the same as the start program counter of the program basic block that needs to be synchronized; and Execute the original instructions of the basic block of the program that needs synchronization.

7. The method according to claim 6, characterized in that Execution management instructions include: Screening threads whose program counters are expected to be executed later are smaller than the program counters of the basic blocks of the program that need to be synchronized, so as to update the mask; Based on the updated mask, a representative thread is selected; using the program counter representing the subsequent expected execution of the thread as the subsequent program counter of the warp; and Jumps to the program counter of the subsequent desired execution of the warp to start executing the subsequent instructions.

8. The method according to claim 1, characterized in that Selecting an active thread as a representative thread includes: selecting an active thread as a representative thread from active threads in the current area based on a round-robin scheduling method.

9. The method according to claim 1, characterized in that Configuring thread local registers within a warp for maintaining a program counter for subsequent expected execution of each thread in the warp includes: Verify whether there is a pattern of mutex locks in the intermediate representation via the compiler front-end and middle-end; and In response to identifying a pattern of mutual exclusion locks in the intermediate representation, thread local registers are configured within the warp for maintaining a program counter that each thread in the warp is subsequently expected to execute.

10. A computing device, characterized in that: include: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a machine, the method according to any one of claims 1 to 9 is performed.

12. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a machine, performs the method according to any one of claims 1 to 9.

Images