Stream processor groups, GPUs, and related methods

Abstract

A stream processor group, GPU and related methods are disclosed. The stream processor group is configured to execute a plurality of thread blocks, wherein each thread block comprises a plurality of thread warps. The stream processor group comprises a plurality of stream processors and a local scheduler. Each stream processor is configured to execute one or more thread warps. The local scheduler comprises a thread warp state table, a thread warp resource detection unit and a thread warp sending unit. The thread warp state table is configured to record allocation status and processing status of each thread warp of each thread block. The thread warp resource detection unit is configured to select all first thread warps of a first thread block and at least one second thread warp of a second thread block according to available hardware resources of the stream processor group and required hardware resources of the thread blocks. The thread warp sending unit is configured to send the first thread warps to idle first stream processors and send the at least one second thread warp to at least one idle second stream processor.

Description

Technical Field

[0001] This application relates to a stream processor group, and more particularly to a stream processor group capable of processing multiple computing cores in parallel to improve computational efficiency. Background Technology

[0002] Because of their parallel computing capabilities, graphics processing units (GPUs) are used not only to render 3D images but also to accelerate artificial intelligence models and big data analysis that require massive parallel computation. Generally, a GPU can comprise multiple streaming multiple processors (SMs), and each SM can comprise multiple streaming processors (SPs). During computation, each SM is typically scheduled to execute one or more thread blocks within kernel code, and each thread block can comprise multiple warps. In this case, the GPU typically schedules multiple streaming processors within a SM to execute multiple threads within one or more warps, on a warp-by-warp basis.

[0003] To improve GPU computing efficiency, some GPUs allow stream processor groups to execute threads from different kernel code simultaneously. However, since each thread block consists of multiple thread bundles, and thread bundles from different kernel code may compete for the same type of hardware resources, scheduling remains quite limited, making it difficult to effectively improve GPU hardware utilization. In other words, how to schedule stream processors within a stream processor group to improve overall computing efficiency remains a pressing problem to be solved in this field. Summary of the Invention

[0004] One of the purposes of this application is to disclose a stream processor group, a GPU, and related methods to solve the above-mentioned problems.

[0005] One embodiment of this application provides a stream processor group for executing multiple thread blocks, wherein each thread block includes multiple warps. The stream processor group includes multiple stream processors and a local scheduler. The local scheduler includes a warp state table, a warp resource detection unit, and a warp sending unit. The warp state table records the allocation and processing status of each warp of the multiple thread blocks. The warp resource detection unit selects all first warps of a first thread block and at least one second warp of a second thread block from the multiple thread blocks, based on the available hardware resources of the stream processor group and the hardware resources required by the multiple thread blocks. The warp sending unit sends the multiple first warps to multiple idle first stream processors among the multiple stream processors, and sends the at least one second warp to at least one idle second stream processor among the multiple stream processors.

[0006] Another embodiment of this application provides a GPU. The GPU includes a plurality of the aforementioned stream processor groups and a global thread block scheduler. The global thread block scheduler is used to allocate multiple thread blocks from a plurality of kernel codes received by the GPU to the plurality of stream processor groups.

[0007] Another embodiment of this application provides a method comprising: a stream processor group receiving multiple thread blocks; recording the allocation and processing status of each thread bundle of the multiple thread blocks; selecting all multiple thread bundles of a first thread block from the multiple thread blocks based on the available hardware resources of the stream processor group and the hardware resources required by the multiple thread blocks; sending the multiple first thread bundles to multiple idle first stream processors in the stream processor group; selecting at least one second thread bundle of a second thread block from the multiple first thread blocks based on the available hardware resources of the stream processor group and the hardware resources required by the multiple thread blocks; and sending the at least one second thread bundle to at least one idle second stream processor in the stream processor group.

[0008] The streaming processor group, GPU, and related methods of this application allow the local scheduler in the streaming processor group to schedule in thread bundles as the smallest unit. Therefore, it can be more flexible in scheduling and increases the probability that the streaming processor group can process thread bundles of different thread blocks at the same time, thereby improving the hardware utilization and computing performance of the streaming processor group. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of a stream processor group according to an embodiment of the present invention.

[0010] Figure 2This is a schematic diagram of a local scheduler scheduling kernel code and its thread blocks according to an embodiment of the present invention.

[0011] Figure 3 This is a flowchart of a thread block scheduling method according to an embodiment of the present invention.

[0012] Figure 4 This is a schematic diagram of a thread bundle state table according to an embodiment of the present disclosure.

[0013] Figure 5 This is a schematic diagram of a graphics processor according to an embodiment of the present disclosure. Detailed Implementation

[0014] The following disclosure provides various implementations or examples that can be used to achieve different features of this disclosure. Specific examples of components and configurations described below are for simplification purposes. It is understood that these descriptions are illustrative only and are not intended to limit the scope of this disclosure. For example, in the following description, forming a first feature on or over a second feature may include, in some embodiments, the first and second features being in direct contact with each other; and may also include, in some embodiments, additional components being formed between the first and second features, such that the first and second features may not be in direct contact. Furthermore, component symbols and / or reference numerals may be reused in multiple embodiments of this disclosure. Such reuse is for the purpose of brevity and clarity and does not in itself represent a relationship between the different embodiments and / or configurations discussed.

[0015] While the numerical ranges and parameters used to define the broader scope of this application are approximate values, the relevant values ​​in the specific embodiments have been presented as precisely as possible. However, any numerical value inevitably contains standard deviations due to individual test methods. Here, "approximately" generally means that the actual value is within plus or minus 10%, 5%, 1%, or 0.5% of a particular value or range. Alternatively, the term "approximately" may mean that the actual value falls within the acceptable standard error of the average, as determined by those skilled in the art to which this application pertains. It is understood that, except for experimental examples, or unless explicitly stated otherwise, all ranges, quantities, values, and percentages used herein (e.g., to describe material usage, duration, temperature, operating conditions, quantity ratios, and the like) are modified with "approximately". Therefore, unless otherwise stated, the numerical parameters disclosed in this specification and the accompanying claims are approximate values ​​and are subject to change as needed. At a minimum, these numerical parameters should be understood as the indicated significant digits and values ​​obtained by applying general rounding. In this context, a range of values ​​is expressed as a distance from one endpoint to the other or between the two endpoints; unless otherwise stated, all ranges of values ​​herein include the endpoints.

[0016] Figure 1 This is a schematic diagram of a stream processor group 100 according to an embodiment of the present invention. The stream processor group 100 may include N stream processors 1101 to 110N and a local scheduler 120, wherein N is an integer greater than 1, and in some embodiments, N may be, for example, but not limited to, 128.

[0017] The stream processor group 100 can receive multiple kernels, each of which can include multiple thread blocks, and each thread block can contain multiple warps. Generally, since multiple threads within the same warp correspond to the same instructions, the local scheduler 120 can schedule warps on a warp-by-warp basis, ensuring that each warp is scheduled to a stream processor for execution. Furthermore, the stream processor group 100 includes hardware resources that can be shared by the stream processors 1101 to 110N. For example, the stream processor group 100 may include registers 130 and memory 140, which the stream processors 1101 to 110N can use to store data required for computation or generated during computation. Since the hardware resources required by each warp may differ, the local scheduler 120 must, during scheduling, allocate suitable warps to idle stream processors based on the available hardware resources of the stream processor group 100 at that time.

[0018] In this embodiment, the local scheduler 120 may include a thread bundle status table 122, a thread bundle resource detection unit 124, and a thread bundle sending unit 126. The thread bundle status table 122 may record the allocation status and processing status of each thread bundle in each thread block received by the stream processor group 100. The thread bundle resource detection unit 124 may select the thread bundle in the thread block to be executed based on the records of the thread bundle status table 122 and the available hardware resources of the stream processor group 100, and send the selected thread bundle to an idle stream processor through the thread bundle sending unit 126 to execute the selected thread bundle.

[0019] Figure 2 This is a schematic diagram illustrating the scheduling of kernel code KA, KB, and their thread blocks by a local scheduler 120 according to an embodiment of the present invention. Figure 2 In this embodiment, the local scheduler 120 needs to schedule thread blocks including P thread blocks TBA1 to TBAP in kernel code KA and Q thread blocks TBB1 to TBBQ in kernel code KB. Each thread block may include multiple thread bundles. For example, thread block TBA1 includes J thread bundles WPA1_1 to WPA1_J, and thread block TBB1 may include K thread bundles WPB1_1 to WPB1_K. In this embodiment, P, Q, K, and J are all integers greater than 1.

[0020] Figure 3 This is a flowchart of a thread block scheduling method 200 according to an embodiment of the present invention. In this embodiment, method 200 may include steps S210 to S290, and the local scheduler 120 may schedule according to method 200. Figure 2 The thread blocks TBA1 to TBAP and TBB1 to TBBQ in the middle.

[0021] In step S210, the stream processor group 100 can receive thread blocks TBA1 to TBAP of kernel code KA and TBB1 to TBBQ of kernel code KB. In step S220, the local scheduler 120 can record the allocation and processing status of each thread bundle in each thread block. For example, since the thread bundles in thread blocks TBA1 to TBAP and TBB1 to TBBQ are all in an unallocated state when the stream processor group 100 first receives them, the local scheduler 120 can record in the thread bundle status table 122 that each thread bundle in thread blocks TBA1 to TBAP and TBB1 to TBBQ is in an "unallocated" state.

[0022] In step S230, the local scheduler 120 can select all thread bundles from thread blocks TBA1 to TBAP and TBB1 to TBBQ based on the available hardware resources of the stream processor group 100 and the hardware resources required by each thread block. For example, the local scheduler 120 can select all thread bundles WPA1_1 to WPA1_J in the first thread block TBA1 from thread blocks TBA1 to TBAP and TBB1 to TBBQ. Then, in step S240, the thread bundle sending unit 126 can send the thread bundles WPA1_1 to WPA1_J to the idle stream processors in stream processors 1101 to 110N, such as stream processors 110X to 110(X+J-1), where X is a positive integer and X+J-1 is less than N. In some embodiments, the stream processor group 100 is located within the GPU, and the GPU's global thread block scheduler can record the hardware resources required by the thread bundles of all kernel code received by the GPU. Therefore, when the global thread block scheduler schedules kernel code KA to the stream processor group 100, it can also transmit the information about the hardware resources required by each thread block and its thread bundle to the stream processor group 100, so that the local scheduler 120 of the stream processor group 100 can schedule each thread bundle according to the hardware resources required by each thread block and its thread bundle. However, this disclosure is not limited thereto. In some other embodiments, the local scheduler 120 can also determine the hardware resources required by each thread bundle based on the content of the received thread block.

[0023] After thread bundles WPA1_1 to WPA1_J are sent to the corresponding stream processors, the local scheduler 120 may, in step S250, update the allocation status of thread bundles WPA1_1 to WPA1_J in the thread bundle status table 122 from "not allocated" to "allocated".

[0024] Furthermore, in step S250, the local scheduler 120 may further record the processing status of thread bundles WPA1_1 to WPA1_J in stream processors 110X to 110(X+J-1). For example, in some embodiments, each thread bundle may have a synchronization point. That is, when the stream processor reaches the synchronization point of a thread bundle, it must wait for all other thread bundles in the same thread block to also reach their respective synchronization points before it can continue to execute the subsequent calculations of each thread bundle. In other words, if the execution progress of thread bundle WPA1_1 is faster and reaches the synchronization point first, the stream processor used to process thread bundle WPA1_1 will pause the execution of thread bundle WPA1_1 and wait for all other thread bundles WPA1_2 to WPA1_J in thread block TBA1 to reach their synchronization points before the stream processor continues to execute the subsequent calculations of thread bundle WPA1_1. In this case, the local scheduler 120 can also record the processing status of thread bundles WPA1_1 to WPA1_J as "stalled-at-sync" or "not stalled" based on the status of the stream processors 110X to 110(X+J-1). That is, when a thread bundle is executed to its synchronization point and the stream processor needs to wait for other thread bundles, the local scheduler 120 can record the processing status of the thread bundle as "stalled-at-sync", while when the thread bundle is being executed by the stream processor, the local scheduler 120 can record the processing status of the thread bundle as "not stalled".

[0025] After thread bundles WPA1_1 to WPA1_J are sent to stream processors 110X to 110(X+J-1) for processing, the local scheduler 120 can then execute step S260 to confirm whether there are still thread blocks to be allocated in the stream processor group 100. In this embodiment, since thread blocks TBA2 to TBAP and TBB1 to TBBQ have not yet been allocated, if hardware resources are sufficient, the local scheduler 120 can also select thread bundles from at least one other thread block and send them to an idle stream processor for processing.

[0026] In some embodiments, thread blocks within the same kernel code may require the same type of hardware resources. Therefore, when stream processor group 100 executes thread block TBA1 in kernel code KA, it may not have sufficient hardware resources to execute thread block TBA2 in kernel code KA. In this case, stream processor group 100 may preferentially select thread blocks TBB1 to TBBQ in kernel code KB, which are different from the kernel code KA to which thread block TBA1 belongs. Furthermore, to ensure that stream processor group 100 can indeed execute the selected thread block, local scheduler 120 will further determine whether stream processor group 100 still has sufficient available hardware resources.

[0027] For example, if thread block TBA1 requires 30% of the capacity of register 130 in stream processor group 100, and thread block TBB1 requires 80% of the capacity of register 130 in stream processor group 100, then when allocating resources in "thread blocks" as the smallest unit, while stream processor group 100 is executing thread block TBA1, only 70% of the capacity of register 130 in stream processor group 100 is available, which is insufficient to provide the hardware resources required to execute thread block TBB1. Therefore, stream processor group 100 must wait for thread block TBA1 to finish executing and release the capacity of register 130 before continuing to execute the existing block TBB1. However, in this embodiment, stream processor group 100 allows the local scheduler 120 to perform scheduling in "thread bundles" as the smallest unit. In other words, in step S270, although the available hardware resources of the stream processor group 100 are insufficient to execute all the thread bundles of the thread block TBB1 after the thread block TBA1 is allocated, if the available hardware resources of the stream processor group 100 are still sufficient to execute some of the thread bundles in the thread block TBB1, the local scheduler 120 can still select some of the thread bundles from the thread block TBB1 according to the available hardware resources of the stream processor group 100 and the hardware resources required by the thread block TBB1, that is, at least one of the thread bundles WPB1_1 to WPB1_K, such as thread bundles WPB1_1 and WPB1_2.

[0028] Next, in step S280, the thread bundle sending unit 120 can send some thread bundles WPB1_1 and WPB1_2 from thread block TBB1 to idle stream processors in stream processors 1101 to 110N, such as stream processors 110Y and 110(Y+1), where Y is a positive integer and Y+1 is less than or equal to N. In this case, the time period during which stream processors 110X to 110(X+J-1) execute thread bundles WPA1_1 to WPA1_J will overlap with the time period during which stream processors 110Y and 110(Y+1) execute thread bundles WPB1_1 and WPB1_2. That is, the stream processor group 100 can execute all thread bundles in thread block TAA1 and some thread bundles in thread block TAB1 in parallel, thereby improving the hardware utilization and overall computing performance of the stream processor group 100.

[0029] After thread bundles WPB1_1 and WPB1_2 are sent to their respective stream processors, the thread bundle sending unit 120 can update the allocation status of thread bundles WPB1_1 and WPB1_2 in the thread bundle status table 122 from "unallocated" to "allocated" in step S290. Furthermore, as previously explained for step S250, in step S290, the local scheduler 120 can also record the processing status of thread bundles WPB1_1 and WPB1_2 in stream processors 110Y and 110(Y+1) as "waiting for synchronization" or "in execution" based on their recorded execution status.

[0030] Figure 4 This is a schematic diagram of a thread bundle state table 122 according to an embodiment of this disclosure. It should be noted that, in this embodiment, for ease of understanding, ... Figure 4 The symbols for each kernel code, thread block, and thread bundle are represented by multiple characters and numbers. However, in some embodiments, each kernel code, thread block, and thread bundle may also be recorded in other appropriate or convenient ways, such as numerical numbers. Similarly, the states such as "allocated," "unallocated," "executing," and "waiting for synchronization" recorded in the thread bundle status table 122 may also be recorded using numbers or Boolean flags.

[0031] like Figure 4As shown, in this embodiment, only some thread bundles WPB1_1 to WPB1_2 in thread block TBB1 are in the "allocated" state, while other thread bundles remain in the "unallocated" state. In this case, when stream processors 110Y and 110(Y+1) reach the synchronization point of thread bundles WPB1_1 to WPB1_2, these bundles must wait until all other thread bundles in thread block TAB1 are allocated and reach their synchronization points before they can end the wait and complete subsequent calculations. In other words, thread bundles WPB1_1 to WPB1_2 may remain in the "waiting for synchronization" state for a long time. During this waiting period, stream processors 110Y and 110(Y+1) used to execute thread bundles WPB1_1 to WPB1_2 will also be in a stopped-and-waiting state, which may reduce the hardware utilization and computational performance of the stream processor group 100.

[0032] To improve the hardware utilization and computing performance of the stream processor group 100, in this embodiment, after thread bundles WPB1_1 to WPB1_2 remain in a "waiting for synchronization" state for a predetermined time, stream processors 110Y and 110(Y+1) can temporarily store the existing computational data of the thread bundles WPB1_1 to WPB1_2 in memory, such as the memory inside stream processors 110Y and 110(Y+1) or the memory 140 in the stream processor group 100, and perform a warp switch operation. That is, stream processors 110Y and 110(Y+1) can be regarded as idle stream processors again, so the warp sending unit 120 can also send the warps in the unallocated thread blocks to stream processors 110Y and 110(Y+1). In this way, the waiting time of the stream processors can be reduced, thereby improving the hardware utilization and reducing the computing performance of the stream processor group 100.

[0033] In this embodiment, each stream processor 1101 to 110N may include a warpscheduler. The warpscheduler can continuously track whether other warps corresponding to a temporarily stored warp have reached their synchronization point. When all warps corresponding to the same warp block have reached their synchronization point, the stream processor can read the temporarily stored data of warps WPB1_1 to WPB1_2 from the memory and continue to execute the subsequent calculations of warps WPB1_1 to WPB1_2.

[0034] Furthermore, in order to reduce the time that thread bundles WPB1_1 to WPB1_2 are in the "waiting for synchronization" state to improve the hardware utilization and computing performance of the stream processor group 100, when the stream processor group 100 has sufficient available hardware resources and multiple thread bundles are in the "waiting for synchronization" processing state in the thread bundle state table 122, the thread bundle resource detection unit 124 can preferentially select the thread block with the most thread bundles in the "waiting for synchronization" state from multiple thread blocks, and send at least one of the thread bundles to the idle processor, so that all thread bundles in the thread block can be executed to their synchronization point as soon as possible.

[0035] For example, such as Figure 4 As shown, although some thread bundles WPB1_1 and WPB1_2 in thread block TBB1 are in the "allocated" state, some thread bundles, such as WPB1_3 to WPB1_K, are still "unallocated". In this case, when thread bundles WPB1_1 and WPB1_2 are executed to the synchronization point, they will enter the "waiting for synchronization" processing state. They must wait for other thread bundles in thread block TBB1, such as the unallocated thread bundles WPB1_3 to WPB1_K, to be allocated and executed to the synchronization point before they can continue execution. Therefore, even if the stream processor group 100 receives new kernel code at this time, in order to avoid the thread bundles WPB1_1 and WPB1_2 waiting for too long, the local scheduler 120 will, if the available hardware resources of the stream processor group 100 are sufficient, prioritize selecting the thread bundles WPB1_3 to WPB1_K that belong to the same thread block TBB1 as the thread bundles WPB1_1 and WPB1_2, and send them to the idle stream processors for processing. Only after all the thread bundles WPB1_1 to WPB1_K of the thread block TBB1 have been allocated will the thread blocks of other kernel code be scheduled.

[0036] Furthermore, if all thread bundles in a thread block have been assigned to a stream processor, the "waiting for synchronization" time for each thread bundle will be relatively short because all thread bundles in the thread block can be executed to the synchronization point. In some embodiments, to shorten the "waiting for synchronization" time for each thread bundle, in step S270, if the available hardware resources of the stream processor group 100 are sufficient to provide the hardware resources required by all thread bundles in thread block TBB1, the local scheduler 120 can preferentially select all thread bundles WPB1_1 to WPB1_K in thread block TBB1, and in step S280 send thread bundles WPB1_1 to WPB1_K to an idle stream processor for execution. In this way, the "waiting for synchronization" time for thread bundles WPB1_1 to WPB1_K can be reduced, thereby improving the computing performance of the stream processor group 100. In other words, when the available hardware resources of the stream processor group 100 are sufficient, the local scheduler 120 should select all the thread bundles in a complete thread block as much as possible to reduce the time that each thread bundle is in "waiting for synchronization", thereby improving the hardware utilization and computing performance of the stream processor group 100.

[0037] In this embodiment, steps S260 to S290 can be executed repeatedly. In step S260, when there are no unallocated thread blocks in the stream processor group 100 or when the stream processor group 100 does not have enough available hardware resources, step S262 can be executed to wait for the stream processor group 100 to receive new thread blocks or to wait for the previously occupied hardware resources to be released.

[0038] Figure 5 This is a schematic diagram of a graphics processing unit (GPU) 30 according to one embodiment of the present disclosure. The GPU 30 includes M stream processor groups 3001 to 300M and a global thread block scheduler 32, where M is an integer greater than 1. In this embodiment, the GPU 30 may be located, for example, in an electronic device, such as a laptop computer or a mobile device, and the central processing unit (CPU) of the electronic device may transfer kernel code to be executed to the GPU 30. Figure 5 As shown, GPU 30 can receive S kernel codes KL1 to KLS, where S is an integer greater than 1, and global thread block scheduler 32 can allocate multiple thread blocks from kernel codes KL1 to KLS to stream processor groups 3001 to 300M. In this embodiment, stream processor groups 3001 to 300M may, for example, have the same structure as stream processor group 100 and can operate according to the same principles.

[0039] The global thread block scheduler 32 may include a kernel code resource status table 322 and a thread block scheduling module 324. The kernel code resource status table 322 can be used to record the hardware resources required by the kernel code KL1 to KLS. That is, the global thread block scheduler 32 can record the hardware resources required by each thread block and thread bundle in each kernel code KL1 to KLS, such as the required register capacity and memory capacity, in the kernel code resource status table 322.

[0040] The thread block scheduling module 324 can schedule each thread block of kernel code KL1 to KLS to stream processor groups 3001 to 300M according to the kernel code resource status table 322. In this embodiment, since each stream processor group 3001 to 300M can schedule multiple thread blocks or thread bundles of kernel code to the stream processors within it through its local scheduler, the thread block scheduling module 324 does not need to consider whether the currently available hardware resources of the stream processor groups 3001 to 300M are sufficient to execute the kernel code to be allocated, and can still continuously schedule each thread block of kernel code KL1 to KLS to the stream processor groups 3001 to 300M.

[0041] Furthermore, in some embodiments, to improve the efficiency of GPU 30 in executing kernel code KL1 to KLS, thread block scheduling module 324 can also allocate kernel code with complementary hardware resource requirements to the same stream processor group, so that the stream processor group has a greater probability of simultaneously executing thread blocks of different kernel code, thereby improving the hardware utilization and computing performance of GPU 30. For example, if kernel code KL1 requires more register capacity but less shared memory capacity, while kernel code KL2 requires more shared memory capacity but less register capacity, it means that the competition between kernel code KL1 and KL2 for the same type of hardware resources is not obvious, and the main hardware resources required by the two are also different. Therefore, the hardware resources required by kernel code KL1 and KL2 can be considered complementary. In this case, after thread block scheduling module 32 schedules the thread block of kernel code KL1 to stream processor group 3001, thread block scheduling module 32 can further prioritize scheduling the thread block of kernel code KL2 to stream processor group 3001. In this way, the Stream Processor 3001 has a greater chance of executing kernel code KL1 and KL2 thread blocks simultaneously, thus improving the hardware utilization of the Stream Processor 3001 and also helping to improve the overall computing performance of GPU 30.

[0042] In summary, the stream processor group, GPU, and related methods provided in the embodiments disclosed herein allow the local scheduler in the stream processor group to schedule in thread bundles as the smallest unit. Therefore, the scheduling is more flexible and the probability of the stream processor group being able to process thread bundles of different thread blocks simultaneously is increased, thereby improving the hardware utilization and computing performance of the stream processor group.

[0043] The foregoing description briefly outlines the features of certain embodiments of this application, enabling those skilled in the art to more fully understand the various forms of this disclosure. Those skilled in the art will readily recognize that this disclosure serves as a basis for designing or modifying other processes and structures to achieve the same objectives and / or advantages as the embodiments described herein. Those skilled in the art should understand that these equivalent embodiments remain within the spirit and scope of this disclosure, and various changes, substitutions, and modifications can be made without departing from the spirit and scope of this disclosure.

Claims

1. A stream processor group, characterized in that, The stream processor group is used to execute multiple thread blocks, each thread block comprising multiple thread bundles, and includes: Multiple stream processors; and The local scheduler includes: A thread bundle state table is used to record the allocation and processing status of each thread bundle of the multiple thread blocks; A thread bundle resource detection unit is configured to, based on the available hardware resources of the stream processor group and the hardware resources required by the plurality of thread blocks, select from the plurality of thread blocks all of the plurality of first thread bundles of a first thread block and at least one second thread bundle of a second thread block; and A thread beam sending unit is used to send the plurality of first thread beams to a plurality of idle first stream processors among the plurality of stream processors, and to send the at least one second thread beam to at least one idle second stream processor among the plurality of stream processors; The time period during which the plurality of first stream processors execute the plurality of first thread bundles overlaps with the time period during which the at least one second stream processor executes the at least one second thread bundle.

2. The stream processor group as described in claim 1, characterized in that, The plurality of first thread bundles correspond to first kernel code, the at least one second thread bundle corresponds to second kernel code, and the thread bundle resource detection unit selects the at least one thread bundle of the second thread block when the available hardware resources of the stream processor group are insufficient to execute all the plurality of thread bundles of the second thread block, but sufficient to execute the at least one second thread bundle of the second thread block after the plurality of first thread bundles are allocated to the plurality of first stream processors.

3. The stream processor group as described in claim 1, characterized in that, After the thread bundle sending unit sends the plurality of first thread bundles to the plurality of first stream processors and sends the at least one second thread bundle to the at least one second stream processor, the thread bundle resource detection unit updates the allocation status of the plurality of first thread bundles and the at least one second thread bundle in the thread bundle status table to allocated.

4. The stream processor group as described in claim 1, characterized in that, After the thread bundle resource detection unit selects all the first thread bundles in the first thread block, when the stream processor group still has sufficient available hardware resources, the thread bundle resource detection unit preferentially selects all the multiple thread bundles in the second thread block to send to the multiple idle stream processors among the multiple stream processors.

5. The stream processor group as described in claim 1, characterized in that, When the at least one second stream processor reaches the synchronization point of the at least one second thread bundle, and at least one other thread bundle in the second thread block other than the at least one second thread bundle has not yet been executed to the synchronization point, the at least one second stream processor stops processing the at least one second thread bundle, and the thread bundle resource detection unit updates the processing status of the at least one second thread bundle in the thread bundle status table to waiting for synchronization.

6. The stream processor group as described in claim 5, characterized in that, After the at least one second thread bundle remains in a waiting-for-synchronization processing state for a predetermined time, the at least one second stream processor temporarily stores the existing computational data of the at least one second thread bundle and performs a thread bundle switching operation to switch to executing another at least one thread bundle.

7. The stream processor group as described in claim 1, characterized in that, When the stream processor group has sufficient available hardware resources, and multiple thread bundles are in a waiting-for-synchronization processing state in the thread bundle state table, the thread bundle resource detection unit preferentially selects at least one thread bundle from the thread block with the most thread bundles in the waiting-for-synchronization state to send to the multiple idle stream processors in the multiple stream processors.

8. A GPU, characterized in that, include: Multiple stream processor groups as described in any one of claims 1 to 7; and A global thread block scheduler is used to allocate multiple thread blocks from multiple kernel codes received by the GPU to the multiple stream processor groups.

9. The GPU as described in claim 8, characterized in that... The global thread block scheduler includes: The kernel code resource status table is used to record the hardware resources required by the various kernel code segments; and The thread block scheduling module is used to schedule the multiple thread blocks of the multiple kernel codes to the multiple stream processor groups based on the kernel code resource status table.

10. The GPU as described in claim 9, characterized in that... After the thread block scheduling module schedules a thread block of the first kernel code among the plurality of kernel codes to the first stream processor group among the plurality of stream processor groups, the thread block scheduling module prioritizes scheduling a thread block of the second kernel code among the plurality of kernel codes, whose required hardware resources are complementary to those of the first kernel code, to the first stream processor group in succession.

11. A method for operating a stream processor group, characterized in that, The stream processor group includes multiple stream processors and a local scheduler, and the method includes: The stream processor group receives multiple thread blocks, wherein each thread block includes multiple thread bundles; The local scheduler records the allocation and processing status of each thread bundle of the multiple thread blocks; The local scheduler selects all the multiple thread bundles of the first thread block from the multiple thread blocks based on the available hardware resources of the stream processor group and the hardware resources required by the multiple thread blocks. The local scheduler sends the plurality of first thread bundles to the plurality of idle first stream processors among the plurality of stream processors; The local scheduler selects at least one thread bundle of the second thread block from the plurality of thread blocks based on the available hardware resources of the stream processor group and the hardware resources required by the plurality of thread blocks; and The local scheduler sends the at least one second thread bundle to at least one idle second stream processor among the plurality of stream processors; The time period during which the plurality of first stream processors execute the plurality of first thread bundles overlaps with the time period during which the at least one second stream processor executes the at least one second thread bundle.

12. The method as described in claim 11, characterized in that... The plurality of first thread bundles correspond to first kernel code, the at least one second thread bundle corresponds to second kernel code, and the step of the local scheduler selecting the at least one thread bundle of the second thread block from the plurality of thread blocks based on the available hardware resources of the stream processor group and the hardware resources required by the plurality of thread blocks includes: After the plurality of first thread bundles are sent to the plurality of first stream processors, when the available hardware resources of the stream processor group are insufficient to execute all of the plurality of thread bundles of the second thread block, but sufficient to execute at least one thread bundle of the second thread block, the local scheduler selects the at least one thread bundle of the second thread block.

13. The method as described in claim 11, characterized in that, further include: After sending the plurality of first thread bundles to the plurality of first stream processors, the local scheduler updates the allocation status of the plurality of first thread bundles to "allocated". and After sending the at least one second thread bundle to the at least one second stream processor, the local scheduler updates the allocation status of the at least one second thread bundle to "allocated".

14. The method as described in claim 11, characterized in that, The local scheduler selects at least one thread bundle of the second thread block from the multiple thread blocks based on the available hardware resources of the stream processor group and the hardware resources required by the multiple thread blocks. This includes selecting all multiple thread bundles of the second thread block when the stream processor group has sufficient available hardware resources.

15. The method as described in claim 11, characterized in that, further include: When the at least one second stream processor reaches the synchronization point of the at least one second thread bundle, and at least one other thread bundle in the second thread block other than the at least one second thread bundle has not yet reached the synchronization point, the at least one second stream processor stops processing the at least one second thread bundle; and The local scheduler updates the processing status of the at least one second thread bundle to waiting for synchronization.

16. The method as described in claim 15, characterized in that, further include: After the at least one second thread bundle remains in a waiting-for-synchronization processing state for a predetermined time, the at least one second stream processor temporarily stores the existing computational data of the at least one second thread bundle. and The at least one second stream processor performs a thread bundle switching operation to switch to executing at least one other thread bundle.

17. The method as described in claim 16, characterized in that, further include: When the stream processor group has sufficient available hardware resources and multiple thread bundles are in a waiting-for-synchronization processing state, the local scheduler preferentially selects from the multiple thread blocks to send at least one thread bundle from the thread block with the most thread bundles in the waiting-for-synchronization state to the multiple idle stream processors.

18. A method for operating a GPU, characterized in that, include: The GPU receives multiple kernel codes; The GPU, based on the hardware resources required by the plurality of kernel codes, schedules a thread block of the first kernel code among the plurality of kernel codes to a stream processor group in the GPU so that the stream processor group executes the method according to any one of claims 11 to 17; and A thread block of a second kernel code, whose required hardware resources are complementary to those of the first kernel code, is sequentially scheduled to the stream processor group.

Images