Thread block based warp scheduling method and GPU scheduling device

Abstract

This invention discloses a thread bundle scheduling method and a GPU scheduling device based on thread blocks. By employing a granular scheduling strategy of prioritizing the oldest thread bundle among thread bundles and round-robin scheduling within thread blocks, it ensures consistent execution progress of thread bundles within the same thread block, reducing synchronization waiting overhead and meeting high-concurrency synchronization requirements. Simultaneously, it achieves progress staggering between different thread blocks, avoiding collective resource contention and collective memory access requests, resolving resource conflicts and memory access latency-induced failures, and perfectly adapting to the load characteristics of large models. The GPU scheduling device can be directly integrated into existing GPU architectures, requiring only the addition of a few functional units to the existing thread bundle scheduling module. It does not require modification of the GPU's computing cores, registers, memory access modules, or other hardware structures, resulting in low hardware modification costs and ease of implementation and deployment.

Description

Technical Field

[0001] This invention belongs to the field of graphics processing unit (GPU) scheduling technology, and more specifically, relates to a thread bundle scheduling method and GPU scheduling device based on thread blocks. Background Technology

[0002] As the core computing platform for parallel computing, the GPU's smallest execution unit is a slot / warp. Each slot / warp contains 32, 64, or 128 threads. The front-end slot scheduler (SS) module of the GPU computing core is responsible for slot scheduling and instruction issuance. With the explosion of large language models, GPUs face severe scheduling challenges in complex operator computations. Efficient slot scheduling strategies must simultaneously meet three major performance requirements: high concurrency synchronization, intensive memory access / computation interleaving, and mitigation of register resource contention.

[0003] High concurrency synchronization requirements: Large model computation involves large-scale thread collaboration. Threads within a workgroup or thread block need to frequently exchange data through synchronization instructions (barriers). If the scheduling strategy cannot balance the execution progress of thread bundles, it will generate huge synchronization waiting overhead. Interleaved memory access and computation requirements: Large model inference includes a large number of KVCache memory access intensive operations and Tensor Core computation intensive operations. The scheduling strategy needs to hide memory access latency by switching thread bundles in different states during long-cycle DMA data loading instructions. Resource contention mitigation requirements: The GPU's internal pipeline is highly parallel. If the execution progress of a large number of thread bundles is too consistent, it will cause bank conflict in the register file (RF) and instantaneous overload of execution units, resulting in frequent pipeline bubbles and reduced computational efficiency.

[0004] Existing thread-beam scheduling strategies in GPU scheduling cannot adapt to the load characteristics of large models, and a new thread-beam scheduling strategy is urgently needed to solve the above problems. Summary of the Invention

[0005] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a thread bundle scheduling method and GPU scheduling device based on thread blocks. This method enables granular hybrid arbitration between and within workgroups, balancing collaboration consistency and schedule diversity. It solves the technical problems of high GPU scheduling synchronization overhead, resource conflicts, and the inability to hide memory access latency under large model loads, while ensuring the integrity of the patent protection scope and adapting to the scheduling needs of various GPU architectures.

[0006] To achieve the above objectives, according to one aspect of the present invention, a thread bundle scheduling method is provided, comprising: filling thread bundles issued to a thread bundle scheduling module into an active queue; the filling rule is: thread bundles within the same thread block are issued sequentially, thread bundles from different thread blocks are not interleaved when issued, and the last thread bundle of each thread block carries an end flag bit; according to the active queue, using the thread block identifier as an index, storing the start position and end position of each thread block in the active queue to form a thread block position table; when a thread bundle meets the instruction issuance condition, setting the ready flag of the corresponding thread bundle in the active queue, generating a global ready mask, the global ready mask reflecting the line in the active queue. The process involves: 1) Detecting the global ready mask; 2) Selecting the highest-ranking ready thread bundle from the active queue and reading its thread block identifier; 3) Using the thread block identifier as the target thread block; 4) Obtaining the target thread block's position mask from the thread block position table to identify the position of all thread bundles within the active queue; 5) Obtaining the final selection mask for all ready thread bundles within the target thread block based on the target thread block's position mask and the global ready mask, indicating the thread bundles within the target thread block that can issue commands; 6) Performing polling arbitration based on the final selection mask of all ready thread bundles within the target thread block to select the corresponding thread bundle for command dispatch.

[0007] In some implementations, the active queue includes state information for each thread bundle, an end flag, a thread block identifier, and a thread bundle identifier; the state information includes ready and not ready states.

[0008] In some implementations, after a thread bundle completes execution, it is removed from the active queue; when a thread bundle is pushed into or removed from the active queue, the start and end positions of the corresponding thread block in the thread block position table are updated in real time.

[0009] In some implementations, the global ready mask is a binary mask, where each bit of the mask corresponds to the ready flag of a thread bundle in the active queue.

[0010] In some implementations, the global ready mask is checked once in each scheduling cycle of the GPU hardware clock to complete the selection of the highest-ranking ready thread bundle.

[0011] In some implementations, the starting and ending positions of the target thread block in the active queue are read from the thread block position table using the thread block identifier of the highest-ranking ready thread bundle as an index, and the position mask of the target thread block is generated; the bit length of the position mask of the target thread block is the same as that of the global ready mask.

[0012] In some implementations, the position mask of the target thread block is bitwise ANDed with the global ready mask to obtain the final selection mask for all ready thread bundles within the target thread block.

[0013] In some implementations, the polling arbitration rule is as follows: if the selected thread block is the same as the previously selected thread block, then polling continues; if a new thread block is selected, then polling starts from the oldest ready thread bundle in the new thread block.

[0014] According to another aspect of the present invention, a GPU scheduling device is provided, including a thread bundle scheduling module; the thread bundle scheduling module includes a queue management unit, a position table management unit, an oldest thread block filtering unit, a mask generation unit, and a polling arbitration unit. The queue management unit is used to fill the active queue with the thread bundles sent to the thread bundle scheduling module, and to set the ready flag of the corresponding thread bundle in the active queue when the thread bundle meets the instruction issuance conditions, and generate a global ready mask. The filling rules are: thread bundles in the same thread block are sent out in order, thread bundles in different thread blocks are not interleaved when they are sent out, and the last thread bundle of each thread block carries an end flag bit. The location table management unit is used to store the start and end positions of each thread block in the active queue, based on the thread block identifier as an index, to form a thread block location table. The oldest thread block filtering unit is used to detect the global ready mask, filter out the ready thread bundle with the highest ranking from the active queue, read its thread block identifier, and obtain the position mask of the corresponding target thread block from the thread block position table based on the thread block identifier. The mask generation unit is used to obtain the final selection mask of all ready thread bundles within the target thread block based on the position mask of the target thread block and the global ready mask; The polling arbitration unit is used to perform polling arbitration based on the final selection mask of all ready thread bundles in the target thread block, and select the corresponding thread bundle for instruction dispatch.

[0015] In some implementations, the active queue includes state information for each thread bundle, an end flag, a thread block identifier, and a thread bundle identifier; the state information includes ready and not ready states.

[0016] According to another aspect of the present invention, a GPU is provided, including the GPU scheduling device described above.

[0017] In summary, the technical solutions conceived by this invention have the following beneficial effects compared with the prior art: 1) Balancing collaboration consistency and progress diversity: Through a granular scheduling strategy of prioritizing the oldest among thread blocks and polling within thread blocks, it ensures consistent execution progress of thread bundles within the same thread block, reduces synchronization waiting overhead, and meets high-concurrency synchronization requirements; it also achieves progress misalignment between different thread blocks, avoids collective resource contention and collective memory access requests, solves resource conflicts and memory access delay hidden failure problems, and perfectly adapts to the load characteristics of large models. 2) Effectively improve GPU computing efficiency: The oldest-first strategy among thread blocks ensures that the thread block that requests resources first releases resources first, improving hardware resource utilization; the polling strategy within thread blocks ensures the number of concurrent thread bundles during task execution, eliminates tailing effects, and reduces pipeline bubbles. Actual tests show that GPU computing efficiency can be improved by more than 40% under large language model inference loads. 3) Lightweight hardware implementation: The new scheduling strategy can be implemented by maintaining only two lightweight data structures, the active queue and the thread block location table, in the scheduling module, combined with simple masking operations and polling arbitration logic. There is no need to make large-scale modifications to the computing cores, registers, memory access modules, etc. of the existing GPU architecture. It is easy to deploy and implement, and can be directly integrated into various GPU products. 4) Adaptable to large-scale computing loads such as large models: It perfectly adapts to the core requirements of high-concurrency synchronization, memory access / computation interleaving, and resource contention mitigation in large-scale model computing, effectively hiding long-cycle memory access latency. It is not only suitable for inference and training of large language models, but also applicable to various GPU general computing scenarios such as deep learning, high-performance computing, and cloud computing. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the thread bundle scheduling method based on thread blocks according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the data structure of the active queue according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the data structure of the thread block location table according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of the GPU scheduling device according to an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of this application. Therefore, the drawings and description are considered exemplary in nature and not restrictive.

[0020] In GPU scheduling, there are two thread scheduling strategies: the oldest-first scheduling algorithm based on age and the round-robin scheduling algorithm.

[0021] The oldest-first-age algorithm prioritizes the thread bundles that have been in the ready queue, executing the longest-lived thread bundles first. While its logic is simple and it can improve resource release efficiency, it suffers from severe synchronization costs and a tailing effect. The progress of different thread bundles within the same workgroup is severely unbalanced, older thread bundles at synchronization points have to wait for new thread bundles for a long time, and resource allocation in the later stages of a task is tilted towards older thread bundles, resulting in an insufficient number of concurrent thread bundles and wasting computational resources.

[0022] The Round-Robin Scheduling algorithm ensures consistent execution progress by polling and arbitrating among all thread bundles, eliminating the tailing effect. However, it suffers from serious resource conflicts and memory access latency issues. In scenarios with homogeneous instructions in large models, all thread bundles compete for hardware resources simultaneously, causing pipeline blockage. Furthermore, all thread bundles initiate memory access requests and enter a waiting state simultaneously. Since there are no ready thread bundles in the scheduler pool that can issue instructions, the long-cycle memory access latency cannot be hidden, resulting in complete hardware idleness.

[0023] Therefore, the oldest-first algorithm sacrifices "cooperation consistency" within a workgroup, while the round-robin algorithm sacrifices "progress diversity" between workgroups. Neither of them can meet the core requirements of GPU scheduling under large model loads.

[0024] The thread block-based thread beam scheduling method of this invention is applied to the GPU's thread beam scheduling module (SS). It employs a granular hybrid arbitration strategy, performing oldest-first scheduling between thread blocks and round-robin scheduling within a thread block. Figure 1 As shown, the thread bundle scheduling method based on thread blocks in this embodiment of the invention specifically includes the following steps: Step S101: Initialize and maintain the active queue (ActiveQueue).

[0025] The thread bundles sent to the thread bundle scheduling module (SS) are filled into the active queue according to the rules. The specific rules for filling the active queue are as follows: thread bundles within the same thread block are sent out in sequence, thread bundles from different thread blocks are not interleaved when they are sent out, and the last thread bundle of each thread block carries a last flag.

[0026] Once the thread bundle has finished executing, it is removed from the active queue.

[0027] The Thread Bundle Scheduling (SS) module includes a queue management unit to maintain an active queue. This active queue is an ordered queue that stores all thread bundle information sent to the SS. For example... Figure 2 As shown, the active queue includes the status information of each thread bundle, the last flag, the thread block identifier (wg_id), and the thread bundle identifier (slot_id). The thread bundle status information includes ready and not ready states.

[0028] Thread bundles are sent to the active queue according to preset rules. These rules are prerequisites for scheduling and have no additional hardware cost, and are guaranteed by the GPU's instruction delivery module.

[0029] Step S103: Initialize and maintain the thread block position table (wg_position_table) based on the active queue.

[0030] SS identifies the thread block identifier (wg_id) and the last flag of each thread bundle. Using the thread block identifier (wg_id) as an index, it stores the start position (wg_start_pos) and end position (wg_end_pos) of each thread block in the Active Queue, i.e., the position of the first and last thread bundles of the thread block. These correspond to the queue indices of the first and last thread bundles in the Active Queue, forming a thread block position table (wg_position_table). Figure 3 As shown.

[0031] When a thread block is pushed into (a new thread block is issued) or removed from the Active Queue (a thread block completes execution), the position table management unit updates the wg_start_pos and wg_end_pos of the corresponding thread block in the wg_position_table in real time to ensure the accuracy of the thread block position information and avoid scheduling errors. If all thread blocks of a certain thread block have completed execution and been removed from the Active Queue, the index information of that thread block is deleted from the thread block position table (wg_position_table).

[0032] Step S105: When the thread bundle meets the instruction issuance condition, set the ready flag (queue_entry_rdy) of the corresponding thread bundle in the Active Queue and generate a global ready mask (queue_rdy_mask).

[0033] Specifically, when a thread bundle completes resource request and meets the instruction issuance conditions, the queue management unit sets the ready flag (queue_entry_rdy) corresponding to that thread bundle in the Active Queue to the first value (e.g., set to 1), indicating that the thread bundle is in a ready state; when the thread bundle does not meet the instruction issuance conditions, the queue management unit sets the ready flag (queue_entry_rdy) corresponding to that thread bundle in the Active Queue to the second value (e.g., set to 0), indicating that the thread bundle is in a not ready state.

[0034] like Figure 2 As shown, the queue management unit generates a global ready mask (queue_rdy_mask) based on the queue_entry_rdy status of all thread bundles. This mask is a binary mask, and each bit of the mask corresponds to the queue_entry_rdy of a thread bundle in the Active Queue. For example, a mask bit value of 1 indicates that the corresponding thread bundle is ready, and a mask bit value of 0 indicates that it is not ready.

[0035] Step S107: The Service Controller (SS) checks the queue_rdy_mask, selects the top-ranked ready thread bundle from the Active Queue, reads its wg_id, and uses the thread block corresponding to wg_id as the target thread block. The SS sets up an oldest thread block filtering unit, which checks the queue_rdy_mask once every clock cycle (i.e., one scheduling cycle of the GPU hardware clock) and selects the thread bundle with queue_entry_rdy set to 1 at the top of the Active Queue, i.e., the oldest ready thread bundle (those that have been in the ready state the longest). After selecting the oldest ready thread bundle, its wg_id is read and recorded as wg_id_x. The thread block corresponding to wg_id_x is the target thread block for this scheduling.

[0036] Step S109: Obtain the location mask of the target thread block.

[0037] Specifically, the oldest thread block filtering unit uses the wg_id of the oldest ready thread bundle as an index to read the wg_start_pos and wg_end_pos of the corresponding thread block in the Active Queue from the wg_position_table, and generates the position mask of the thread block. The bit length of the mask is the same as that of the queue_rdy_mask, with only the bit interval from wg_start_pos to wg_end_pos set to 1 and the rest set to 0, which is used to identify the position of the target thread block in all thread bundles in the Active Queue.

[0038] Step S111: Generate the final selection mask (final_sel_msk) for the target thread block.

[0039] Specifically, the position mask of the target thread block is ANDed with the global ready mask queue_rdy_mask to obtain the final selection mask final_sel_msk of all ready thread bundles within the target thread block.

[0040] SS includes a mask generation unit that performs a bitwise AND operation between the target thread block's position mask and the global ready mask `queue_rdy_mask` to obtain the final selection mask `final_sel_msk`. For example, the positions where a bit in `final_sel_msk` is 1 represent the positions of all ready thread bundles within the target thread block in the Active Queue. This final selection mask allows for precise filtering of thread bundles within the target thread block that can issue instructions, avoiding a full search of the Active Queue and reducing hardware timing overhead.

[0041] Step S113: SS performs polling arbitration based on final_sel_msk and selects the corresponding thread bundle for instruction dispatch.

[0042] If the selected thread block is the same as the previously selected thread block, the polling continues; if a new thread block is selected, the polling starts from the oldest ready thread bundle in the new thread block.

[0043] In SS, a polling arbitration unit is set up. Based on final_sel_msk, polling arbitration is performed on the ready thread bundles in the target thread block to select the corresponding thread bundle for instruction issuance and dispatch.

[0044] The specific rules for polling arbitration are as follows: (1) If the thread block identifier of the target thread block in this scheduling is the same as the thread block identifier of the previous scheduling, then continue to select the next ready thread bundle in the polling order within the thread block; (2) If the target thread block of this scheduling is a new thread block (different from the thread block identifier of the previous scheduling), then polling starts from the oldest ready thread bundle in the new thread block (the thread bundle corresponding to the first bit with a value of 1 in final_sel_msk).

[0045] After a thread completes instruction dispatch, if the instruction is executed successfully, it is removed from the Active Queue, and the position table management unit updates wg_position_table synchronously. If the instruction is not executed successfully and is still in a ready state, it is retained in the Active Queue, waiting for the next scheduling.

[0046] The thread bundle scheduling method based on thread blocks in this invention is applied to the thread bundle scheduling module (SS) of a GPU. This GPU can be used in general computing scenarios for large language model inference. The smallest execution unit of the GPU is a thread bundle (Slot), and each thread bundle contains, for example, 32, 64, or 128 threads. Several thread bundles form a thread block (ThreadBlock), and each thread block contains, for example, 1 to 8 thread bundles. The SS is responsible for scheduling and issuing instructions for thread bundles. It adopts a granular hybrid arbitration strategy of prioritizing the oldest among thread blocks and polling within thread blocks, which perfectly avoids the defects of existing priority and polling strategies. It not only ensures the consistency of the execution progress of thread bundles within the same thread block, reducing synchronization waiting overhead by more than 60% and meeting the high concurrency synchronization requirements, but also realizes the progress misalignment between different thread blocks, avoids collective resource contention and collective memory access requests, solves the problems of resource conflict and memory access latency hiding failure, ensures progress diversity, improves memory access latency hiding efficiency by more than 70%, and perfectly adapts to the load characteristics of large models.

[0047] This invention also provides a GPU scheduling device for implementing the above scheduling method. The device is integrated into the front end of the GPU's computing core. Its main component is a thread beam scheduling module (SS), which includes a queue management unit, a position table management unit, an oldest thread block filtering unit, a mask generation unit, and a polling arbitration unit. The functions of each unit are consistent with the corresponding functions in the above steps. The units work together to achieve granular thread beam scheduling.

[0048] The queue management unit maintains the active queue, filling the issued thread bundles into the active queue according to rules. When a thread bundle meets the issuance conditions, it sets the corresponding ready flag and generates a global ready mask. The rules are as follows: thread bundles within the same thread block are issued sequentially; thread bundles from different thread blocks are not interleaved; and the last thread bundle in each thread block carries a last flag. The active queue stores the status information, last flag, wg_id, and slot_id for each thread bundle. The status information includes ready and not ready states.

[0049] The position table management unit is used to identify the wg_id and last flag of the thread bundle, maintain and update the thread block position table (wg_position_table); the wg_position_table uses wg_id as an index to store the start and end positions of each thread block in the Active Queue, and is updated in real time when a thread bundle is pushed into or pushed out of the Active Queue.

[0050] The oldest thread block filtering unit is used to detect the global ready mask, filter out the oldest ready thread bundle from the Active Queue, read its wg_id, and obtain the position mask of the corresponding thread block from the wg_position_table based on the wg_id.

[0051] The mask generation unit performs a bitwise AND operation between the position mask of the thread block and the global ready mask to generate the final selection mask for the ready thread bundles within the thread block. The oldest thread block filtering unit performs a global ready mask check and the filtering of the oldest ready thread bundle once per cycle.

[0052] The polling arbitration unit is used to perform polling arbitration based on the final selection mask and select a thread bundle for instruction dispatch. If the selected thread block is the same as the previous one, polling continues. If a new thread block is switched, polling starts from the oldest ready thread bundle in that thread block.

[0053] Furthermore, such as Figure 4As shown, the queue management unit is connected to the position table management unit and the oldest thread block filtering unit, respectively. It transmits the thread block identifier (wg_id) and last flag of the thread bundle to the position table management unit, and transmits the global ready mask (queue_rdy_mask) to the oldest thread block filtering unit. The position table management unit is connected to the oldest thread block filtering unit, providing indexing and query services for the thread block position table (wg_position_table) to the oldest thread block filtering unit. The oldest thread block filtering unit is connected to the mask generation unit, transmitting the position mask of the target thread block to the mask generation unit. The queue management unit is also connected to the mask generation unit, transmitting the global ready mask (queue_rdy_mask) to the mask generation unit. The mask generation unit is connected to the polling arbitration unit, transmitting the final selection mask (final_sel_msk) to the polling arbitration unit. The polling arbitration unit is the instruction dispatch execution unit, connected to the GPU's instruction issuing module, used to dispatch the selected thread bundle instructions to the GPU's execution unit through the instruction issuing module.

[0054] The GPU scheduling device of this invention can be directly integrated into the existing GPU architecture. It only requires adding the above five functional units to the original SS module. There is no need to modify the hardware structure of the GPU, such as the computing core, registers, and memory access module. The hardware modification cost is low, it is easy to implement and deploy, and it can be adapted to GPU products from various manufacturers such as NVIDIA and AMD.

[0055] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0056] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0057] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more (two or more) executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0058] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0059] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0060] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0061] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A thread beam scheduling method, characterized in that, include: The thread bundles sent to the thread bundle scheduling module are filled into the active queue. The filling rules are as follows: thread bundles within the same thread block are sent out in order, thread bundles from different thread blocks are not interleaved when they are sent out, and the last thread bundle of each thread block carries an end flag. Based on the active queue, using the thread block identifier as an index, store the start and end positions of each thread block in the active queue to form a thread block position table; When a thread bundle meets the instruction issuance conditions, the ready flag of the corresponding thread bundle in the active queue is set, and a global ready mask is generated. The global ready mask reflects the state information of the thread bundles in the active queue. Detect the global ready mask, select the ready thread bundle with the highest ranking from the active queue, read its thread block identifier, and take the thread block corresponding to the thread block identifier as the target thread block. Based on the thread block location table, obtain the location mask of the target thread block, which is used to identify the position of all thread bundles of the target thread block in the active queue; Based on the location mask and global ready mask of the target thread block, the final selection mask of all ready thread bundles within the target thread block is obtained, which is used to indicate the thread bundles within the target thread block that can issue commands. Based on the final selection mask of all ready thread bundles within the target thread block, a polling arbitration is performed to select the corresponding thread bundle for instruction dispatch.

2. The thread beam scheduling method as described in claim 1, characterized in that, The active queue includes the status information, end flag, thread block identifier, and thread bundle identifier for each thread bundle; the status information includes ready and not ready states.

3. The thread beam scheduling method as described in claim 1, characterized in that, Once a thread bundle completes its execution, it is removed from the active queue. When a thread bundle is pushed into or removed from the active queue, the start and end positions of the corresponding thread block in the thread block position table are updated in real time.

4. The thread beam scheduling method as described in claim 1, characterized in that, The global ready mask is a binary mask, and each bit of the mask corresponds to the ready flag of a thread bundle in the active queue.

5. The thread beam scheduling method as described in claim 1, characterized in that, The global ready mask is checked once in each scheduling cycle of the GPU hardware clock to complete the selection of the ready thread bundle with the highest ranking.

6. The thread beam scheduling method as described in claim 1, characterized in that, Using the thread block identifier of the highest-ranking ready thread bundle as an index, the start and end positions of the target thread block in the active queue are read from the thread block position table to generate the position mask of the target thread block; the bit length of the position mask of the target thread block is the same as that of the global ready mask.

7. The thread beam scheduling method as described in claim 1, characterized in that, Perform a bitwise AND operation between the position mask of the target thread block and the global ready mask to obtain the final selection mask for all ready thread bundles within the target thread block.

8. The thread beam scheduling method as described in claim 1, characterized in that, The polling arbitration rules are as follows: if the selected thread block is the same as the previously selected thread block, then polling continues; if a new thread block is selected, then polling starts from the oldest ready thread bundle in the new thread block.

9. A GPU scheduling device, characterized in that, It includes a thread bundle scheduling module; the thread bundle scheduling module includes a queue management unit, a position table management unit, an oldest thread block filtering unit, a mask generation unit, and a polling arbitration unit; The queue management unit is used to fill the thread bundles sent to the thread bundle scheduling module into the active queue, and when the thread bundle meets the instruction issuance conditions, set the ready flag of the corresponding thread bundle in the active queue and generate a global ready mask. The filling rules are as follows: thread bundles within the same thread block are issued in sequence, thread bundles from different thread blocks are not interleaved when issued, and the last thread bundle of each thread block carries an end flag bit; The location table management unit is used to store the start and end positions of each thread block in the active queue, using the thread block identifier as an index, to form a thread block location table. The oldest thread block filtering unit is used to detect the global ready mask, filter out the ready thread bundle with the highest ranking from the active queue, read its thread block identifier, and obtain the position mask of the corresponding target thread block from the thread block position table based on the thread block identifier. The mask generation unit is used to obtain the final selection mask of all ready thread bundles within the target thread block based on the position mask of the target thread block and the global ready mask. The polling arbitration unit is used to perform polling arbitration based on the final selection mask of all ready thread bundles in the target thread block, and select the corresponding thread bundle for instruction dispatch.

10. The GPU scheduling device as described in claim 9, characterized in that, The active queue includes the status information, end flag, thread block identifier, and thread bundle identifier for each thread bundle; the status information includes ready and not ready states.

11. A GPU, characterized in that, Includes the GPU scheduling device as described in claim 9 or 10.

Images