Thread block scheduling method, system and device based on thread beam scheduling algorithm, and medium
By dynamically adjusting the number of thread blocks and policy optimization based on thread bundle scheduling algorithm, the problem that traditional scheduling strategies cannot cope with different workloads is solved, the performance and adaptability of GPGPU is improved, and resource competition and execution delay is reduced.
Patent Information
- Application Number
- CN202411127089.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-16
- Publication Date
- 2024-11-15
AI Technical Summary
The traditional GPGPU thread block scheduling strategy cannot effectively deal with the different demands of thread blocks by different workloads, resulting in unstable performance and resource competition leading to performance degradation.
Using a method based on thread bundle scheduling algorithm, the number of thread blocks is dynamically adjusted through GTO and scheduling strategies that perceive thread block sequence, combined with Round-Robin and GTO algorithms, the performance and memory access efficiency are optimized using LCS and BCS strategies.
It improves the scheduling efficiency of GPGPU, reduces resource competition, enhances adaptability to different application scenarios, improves computing performance and memory access efficiency, and reduces execution delay.
Smart Images

Figure CN118963962A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of GPGPU thread block scheduling, and to a thread block scheduling method, system, device, and medium based on a thread warp scheduling algorithm. Background Art
[0002] At present, general-purpose parallel graphics processing units (GPGPUs), as a key technology in the field of high-performance computing, are widely used in scientific computing, data analysis, machine learning and other fields. The core advantage of GPGPU lies in its huge computing resources, which can execute thousands of threads simultaneously, thus realizing large-scale parallel computing tasks.
[0003] In the GPGPU programming model, threads are organized into thread bundles (warps), which are further combined into thread blocks. Threads within a thread block are executed on the same core, forming a two-level scheduling system within the GPGPU. The first level is the warp scheduler, which is responsible for deciding which warp will be executed; the second level is the thread block scheduler, which is responsible for assigning thread blocks to cores. This two-level scheduling mechanism is crucial to optimizing the performance of GPGPU.
[0004] In the process of implementing the embodiments of the present disclosure, it is found that there are at least the following problems in the related art: Experimental results show that simply increasing the number of thread blocks does not always improve performance. For some workloads, performance improves with the number of thread blocks; while for other workloads, too many thread blocks can lead to performance degradation, possibly due to resource contention. Therefore, traditional scheduling strategies, such as simply assigning the maximum number of thread blocks to each core, are not always optimal.
[0005] Thread block scheduling is the process of deciding which thread blocks are assigned to the GPGPU core for execution. How to stably and dynamically obtain the load situation and reasonably allocate the number of thread blocks to improve system performance is a very important topic. Existing scheduling strategies may not be able to effectively cope with the different requirements of different workloads for the number of thread blocks. A smarter and more flexible scheduling method is needed to optimize the performance of GPGPU.
[0006] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention
[0007] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical elements or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.
[0008] The embodiments of the present disclosure provide a thread block scheduling method and system, device, and medium based on a thread warp scheduling algorithm, which solve the problems in the prior art and improve the scheduling efficiency and performance of GPGPU.
[0009] In some embodiments, the method comprises: The system is powered on and the preset maximum number of thread blocks is allocated to each core; The CPU controller obtains the kernel layout scheme according to the system configuration information, and selectively starts the thread warp scheduling strategy that perceives the thread block order according to the kernel layout mode; The core executes instructions using the GTO warp scheduling strategy and measures the number of instructions executed by each thread block; The first thread block completes the execution process and enters the restriction phase. The CPU controller reads the number of instructions executed by the first thread block stored in the register in the kernel and calculates the number of new thread blocks. ,The CPU controller retransmits the calculated new number of thread blocks to each core as a system parameter; During the deferred execution phase, the kernel uses the allocated Active thread blocks continue to execute until the thread block completes.
[0010] Preferably, the kernel layout scheme includes 1D layout and 2D layout. If the kernel adopts 2D layout, the thread warp scheduling strategy that perceives the thread block order is started. If the kernel adopts 1D layout, the GTO thread warp scheduling strategy is directly adopted.
[0011] Preferably, the thread block order-aware warp scheduling strategy combines two warp scheduling algorithms: Round-Robin and GTO.
[0012] Preferably, the calculation formula for the new number of thread blocks is: , in, is the total number of instructions, is the number of instructions of the first completed thread block, the total number of instructions is the total number of instructions issued in all thread blocks of the core, and the number of instructions of the first completed thread block is the maximum number of instructions executed by each thread block.
[0013] Preferably, the LCS policy is also included to optimize performance by limiting the maximum number of thread blocks allocated to each core.
[0014] Preferably, a BCS strategy is also included to allocate consecutive thread blocks to the same core, thereby improving the locality of memory access and cache utilization.
[0015] In some embodiments, the system comprises: The thread block scheduler is used to allocate a preset maximum number of thread blocks to each core; A configuration information processor is used to obtain a layout plan of the kernel and start a thread warp scheduling strategy that is aware of the thread block order according to the layout plan; The warp execution unit executes instructions using the GTO warp scheduling strategy and measures the number of instructions executed by each thread block; A thread block number update unit, used to calculate and update the number of thread blocks for each core; An execution control unit for continuing execution with the updated number of thread blocks in the limit phase and reducing the waiting time of thread blocks on the cores in the delay execution phase.
[0016] In some embodiments, the apparatus includes: a processor and a memory storing program instructions, and the processor is configured to execute the thread block scheduling method based on the thread warp scheduling algorithm when running the program instructions.
[0017] In some embodiments, the medium includes: program instructions stored therein, and when the program instructions are run, the thread block scheduling method based on the thread warp scheduling algorithm as described above is executed.
[0018] The embodiments of the present disclosure provide a thread block scheduling method, system, device, and medium based on a thread warp scheduling algorithm, which can achieve the following technical effects: By combining the GTO algorithm and the scheduling strategy that perceives the thread block order, the present invention optimizes the scheduling process of the thread block and improves the scheduling efficiency of the GPGPU.
[0019] By limiting the maximum number of thread blocks of each core and adjusting it dynamically, the present invention reduces resource competition and avoids performance degradation caused by too many thread blocks.
[0020] The present invention can dynamically adjust the number of thread blocks according to different workloads, thereby enhancing the adaptability of GPGPU to different application scenarios.
[0021] Through the BCS strategy, the present invention improves the locality of memory access and cache utilization, thereby improving the overall computing performance.
[0022] In the delayed execution stage, by reducing the waiting time of the thread block on the core, the present invention reduces the overall execution delay and speeds up the task completion speed.
[0023] The LCS strategy optimizes the allocation of core resources and avoids resource waste by limiting the number of thread blocks.
[0024] The BCS strategy improves the continuity of thread execution by allocating consecutive thread blocks to the same core, which helps improve execution efficiency.
[0025] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein: Figure 1 It is a schematic flow chart of the method of the present invention; Figure 2 This is a schematic diagram of the GPGPU architecture; Figure 3 It is a schematic diagram of the implementation process of the method provided by the embodiment of the present disclosure; Figure 4 is a schematic diagram of the thread block layout network; Figure 5 This is a schematic diagram of the traditional thread block round-robin scheduling method; Figure 6 It is a schematic diagram of the thread block continuous block scheduling method proposed by this invention; Figure 7 It is a schematic diagram of a device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION
[0027] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.
[0028] The terms "first", "second", etc. in the specification and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so as to describe the embodiments of the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.
[0029] Unless otherwise stated, the term "plurality" means two or more.
[0030] Example 1 like Figure 1 As shown, a thread block scheduling method based on a thread warp scheduling algorithm includes: S1: The system is powered on and the preset maximum number of thread blocks is allocated to each core.
[0031] S2: The CPU controller obtains the kernel layout scheme according to the system configuration information, and selectively starts the thread warp scheduling strategy that is aware of the thread block order according to the kernel layout mode.
[0032] S3: The core executes instructions using the GTO warp scheduling policy and measures the number of instructions executed by each thread block.
[0033] S4: The first thread block completes the execution process and enters the restriction phase. The CPU controller reads the number of instructions executed by the first thread block stored in the register in the kernel and calculates the number of new thread blocks. ,The CPU controller retransmits the calculated new number of thread blocks to each core as a system parameter.
[0034] S5: During the delayed execution phase, the kernel uses the allocated Active thread blocks continue to execute until the thread block completes.
[0035] As a refinement of the above embodiment, the layout scheme of the kernel includes a 1D layout and a 2D layout. If the kernel adopts a 2D layout, a thread warp scheduling strategy that perceives the thread block order is started. If the kernel adopts a 1D layout, a GTO thread warp scheduling strategy is directly adopted.
[0036] As a refinement of the above embodiment, the thread block sequence-aware thread warp scheduling strategy combines the Round-Robin and GTO thread warp scheduling algorithms, and can sense the "continuous blocks" of thread blocks assigned to the core, thereby more effectively scheduling warps.
[0037] As a refinement of the above embodiment, the calculation formula for the new number of thread blocks is: , in, is the total number of instructions, is the number of instructions of the first completed thread block, the total number of instructions is the total number of instructions issued in all thread blocks of the core, and the number of instructions of the first completed thread block is the maximum number of instructions executed by each thread block.
[0038] As a refinement of the above embodiment, a (Lazy CTA Scheduling) strategy is also included to optimize performance by limiting the maximum number of thread blocks assigned to each core. This method uses the GTO warp scheduler to determine the optimal number of thread blocks and makes decisions only by measuring the number of instructions issued.
[0039] As a refinement of the above embodiment, a BCS (Block CTA Scheduling) strategy is also included to allocate consecutive thread blocks to the same core, thereby improving the locality of memory access and cache utilization.
[0040] This embodiment uses a 1-dimensional kernel layout as an example to describe the scheduling method in detail: After the system is powered on, a preset maximum number of thread blocks is allocated to each core.
[0041] The CPU controller obtains the kernel layout plan based on the system configuration information. If the kernel adopts a 1-D layout, each core uses the GTO thread bundle scheduling strategy to execute instructions and measures the number of instructions executed by each thread block. When all threads in the first thread block are executed, the current instruction accumulation value is saved in the register, and the completion signal is pulled high and reported to the CPU controller.
[0042] After the first thread block completes its execution, the system enters the limiting phase. The CPU reads the number of instructions executed by the first thread block stored in the register in the kernel and calculates the number of new thread blocks according to the following formula: : , in, is the total number of instructions, is the number of instructions of the first completed thread block, the total number of instructions is the total number of instructions issued in all thread blocks of the core, and the number of instructions of the first completed thread block is the maximum number of instructions executed by each thread block.
[0043] After the limit phase is completed, each core is assigned a new number of thread blocks. During the deferred execution phase, the kernel will only use the allocated Active thread blocks continue to execute until these thread blocks are completed. By delaying the execution strategy, the waiting time of thread blocks on the core is reduced. It ensures that thread blocks start executing on the core as soon as possible, thereby reducing the overall execution latency.
[0044] This embodiment uses a 2-dimensional kernel layout as an example to describe the scheduling method in detail: After the system is powered on, a preset maximum number of thread blocks is allocated to each core.
[0045] The CPU controller obtains the kernel layout scheme based on the system configuration information. If the kernel adopts a two-dimensional layout, the thread warp scheduling strategy that perceives the thread block order is started. This strategy combines the RR (Round-Robin) and GTO thread warp scheduling algorithms. In order to effectively utilize the locality between thread blocks, the thread warps are scheduled between two thread warps of adjacent thread blocks or within the same block in a polling manner.
[0046] Each core uses the GTO thread bundle scheduling strategy to execute instructions and measures the number of instructions inst executed by each thread block. When all threads in the first thread block are executed, the current instruction accumulation value is saved in the register, and the completion signal is pulled high and reported to the CPU controller.
[0047] After the first thread block completes its execution, the system enters the limiting phase. The CPU reads the number of instructions executed by the first thread block stored in the register in the kernel and calculates the number of new thread blocks according to the following formula: : , in, is the total number of instructions, is the number of instructions of the first completed thread block, the total number of instructions is the total number of instructions issued in all thread blocks of the core, and the number of instructions of the first completed thread block is the maximum number of instructions executed by each thread block.
[0048] After the limit phase is completed, each core is assigned a new number of thread blocks. During the deferred execution phase, the kernel will only use the allocated Active thread blocks continue to execute until these thread blocks are completed. By delaying the execution strategy, the waiting time of thread blocks on the core is reduced. It ensures that thread blocks start executing on the core as soon as possible, thereby reducing the overall execution latency.
[0049] Example 2 A thread block scheduling system based on a thread warp scheduling algorithm, comprising: The thread block scheduler is used to allocate a preset maximum number of thread blocks to each core.
[0050] The configuration information processor is used to obtain the layout plan of the kernel and start the thread warp scheduling strategy that is aware of the thread block order according to the layout plan.
[0051] The warp execution unit executes instructions using the GTO warp scheduling policy and measures the number of instructions executed by each thread block.
[0052] The thread block number update unit is used to calculate and update the number of thread blocks for each core.
[0053] An execution control unit for continuing execution with the updated number of thread blocks in the limit phase and reducing the waiting time of thread blocks on the cores in the delay execution phase.
[0054] Example 3 Combination Figure 7 As shown, the embodiment of the present disclosure provides a thread block scheduling device 300 based on a thread warp scheduling algorithm, including a processor 304 and a memory 301. Optionally, the device may also include a communication interface 302 and a bus 303. The processor 304, the communication interface 302, and the memory 301 may communicate with each other through the bus 303. The communication interface 302 may be used for information transmission. The processor 304 may call the logic instructions in the memory 301 to execute the thread block scheduling method based on the thread warp scheduling algorithm of the above embodiment.
[0055] In addition, the logic instructions in the memory 301 described above can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.
[0056] The memory 301 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor 304 executes the function application and data processing by running the program instructions / modules stored in the memory 301, that is, the thread block scheduling method based on the thread warp scheduling algorithm in the above embodiment is implemented.
[0057] The memory 301 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 301 may include a high-speed random access memory and may also include a non-volatile memory.
[0058] An embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute the thread block scheduling method based on the thread warp scheduling algorithm.
[0059] The computer-readable storage medium mentioned above may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.
[0060] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiment of the present disclosure. The aforementioned storage medium may be a non-transient storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and other media that can store program codes, or a transient storage medium.
[0061] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible changes. Unless explicitly required, separate components and functions are optional, and the order of operations may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and the claims, unless the context clearly indicates, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of one or more associated listings. In addition, when used in this application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, method or device including the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.
[0062] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. The technicians may clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.
[0063] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically separately, or two or more units may be integrated in one unit.
[0064] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.
Claims
1. A thread block scheduling method based on thread warp scheduling algorithm, It is characterized in that The following steps are involved: The system is powered on and the preset maximum number of thread blocks is allocated to each core; The CPU controller obtains the kernel layout scheme according to the system configuration information, and selectively starts the thread warp scheduling strategy that perceives the thread block order according to the kernel layout mode; The core executes instructions using the GTO warp scheduling strategy and measures the number of instructions executed by each thread block; The first thread block completes the execution process and enters the restriction phase. The CPU controller reads the number of instructions executed by the first thread block stored in the register in the kernel and calculates the number of new thread blocks. ,The CPU controller retransmits the calculated new number of thread blocks to each core as a system parameter; During the deferred execution phase, the kernel uses the allocated Active thread blocks continue to execute until the thread block completes.
2. The thread block scheduling method based on the thread warp scheduling algorithm according to claim 1, It is characterized in that The layout scheme of the kernel includes 1D layout and 2D layout. If the kernel adopts 2D layout, the thread warp scheduling strategy that perceives the thread block order is started. If the kernel adopts 1D layout, the GTO thread warp scheduling strategy is directly adopted.
3. The thread block scheduling method based on the thread warp scheduling algorithm according to any one of claims 1 or 2, It is characterized in that The thread block order-aware thread warp scheduling strategy combines two thread warp scheduling algorithms, Round-Robin and GTO.
4. The thread block scheduling method based on the thread warp scheduling algorithm according to claim 1, It is characterized in that The calculation formula for the new number of thread blocks is: , in, is the total number of instructions, is the number of instructions of the first completed thread block, the total number of instructions is the total number of instructions issued in all thread blocks of the core, and the number of instructions of the first completed thread block is the maximum number of instructions executed by each thread block.
5. The thread block scheduling method based on the thread warp scheduling algorithm according to claim 1, It is characterized in that Also included is the LCS policy to optimize performance by limiting the maximum number of thread blocks assigned to each core.
6. The thread block scheduling method based on the thread warp scheduling algorithm according to claim 1, It is characterized in that It also includes the BCS strategy to assign consecutive thread blocks to the same core, improving memory access locality and cache utilization.
7. A thread block scheduling system based on a thread warp scheduling algorithm for executing the method according to any one of claims 1 to 6, It is characterized in that include: The thread block scheduler is used to allocate a preset maximum number of thread blocks to each core; A configuration information processor is used to obtain a layout plan of the kernel and start a thread warp scheduling strategy that is aware of the thread block order according to the layout plan; The warp execution unit executes instructions using the GTO warp scheduling strategy and measures the number of instructions executed by each thread block; A thread block number update unit, used to calculate and update the number of thread blocks for each core; An execution control unit for continuing execution with the updated number of thread blocks in the limit phase and reducing the waiting time of thread blocks on the cores in the delay execution phase.
8. A thread block scheduling device based on a thread warp scheduling algorithm, comprising a processor and a memory storing program instructions, It is characterized in that The processor is configured to execute the thread block scheduling method based on the thread warp scheduling algorithm according to any one of claims 1 to 6 when running the program instructions.
9. A storage medium storing program instructions, It is characterized in that When the program instructions are executed, the thread block scheduling method based on the thread warp scheduling algorithm according to any one of claims 1 to 6 is executed.
Citation Information
Cited By
Prefetch optimization method and system of GPGPU and storage medium
CN119415445A
Operator calculation method, electronic equipment and storage medium
CN121052300A
