Live round-robin scheduling methods, systems, electronic devices, and storage media for thread blocks.

Abstract

This invention provides a method, system, electronic device, and storage medium for live round-robin scheduling of thread blocks, belonging to the field of GPGPU thread scheduling and execution. The method includes: when a thread block's thread bundle encounters a long-delay memory access operation, receiving feedback from the thread bundle scheduler, matching the corresponding thread block entry using the feedback ID number, and decrementing the liveness value by 1. When the liveness value is zero, the thread block is deactivated, and the execution context information of the thread block is stored in shared memory. The shared memory stores data for multiple thread blocks and the round-robin context information for each thread block. Simultaneously, thread blocks with liveness values ​​are selected for activation and sent to the pipeline for scheduling. When the thread block is reactivated, its context information is read from the shared memory for continued execution. By polling and switching live thread blocks, the latency of long-delay instructions is masked, improving the GPGPU's operating efficiency.

Description

Technical Field

[0001] This invention belongs to the field of GPGPU thread scheduling and execution technology, specifically relating to a method, system, electronic device, and storage medium for live round-robin scheduling of thread blocks. Background Technology

[0002] GPGPU can execute thousands of threads simultaneously. The GPGPU kernel is divided into large thread blocks, and it supports rapid context switching between warps, allowing other warps to execute while one warp is paused, thus hiding instruction latency. To further support fast context switching, GPGPU uses a large register file to store the contexts of multiple concurrent warps, eliminating the need for context restoration and saving.

[0003] However, the number of CTAs that can execute concurrently on the SM is determined by various hardware resource limitations, such as the maximum number of thread blocks supported by the hardware, the maximum number of threads supported by the hardware, the size of the register file, and the size of shared memory. If the cumulative resource requirements of all allocated CTAs reach any of these limits, no more CTAs can be scheduled to the SM, even if other resources are still available. Statistically, under the above constraints, the maximum number of thread blocks supported by the hardware and the maximum number of threads supported by the hardware are the main limiting factors affecting thread-level parallelism on GPGPUs. However, simply increasing the maximum number of thread blocks and threads scheduled by the hardware will significantly increase the hardware resources of the circuit. It requires additional scheduling support, such as a larger instruction buffer, scoreboard, and SIMT stack. At the same time, the complexity of the scheduler will also increase, the search time for warps that can be issued will increase, and thus increase the issue latency. Summary of the Invention

[0004] The purpose of this invention is to provide a method, system, electronic device, and storage medium for live round-robin scheduling of thread blocks. This method utilizes the shared memory and register file capacity on the SM to allocate as many thread blocks as possible to the SM until the shared memory or register file is full. It also uses round-robin switching of live thread blocks to mask the latency of long-delay instructions. Furthermore, the context management design is relatively simple, which improves the operating efficiency of the GPGPU.

[0005] In a first aspect, embodiments of this application provide a live round-robin scheduling method for thread blocks, including:

[0006] When a thread block encounters a long-delay memory access operation, it receives feedback from the thread block scheduler, matches the corresponding entry of the thread block with the feedback ID number, and decrements the activity value by 1. When the activity value is zero, the thread block is deactivated, and the execution context information of the thread block is stored in the shared memory. At the same time, a thread block with an activity value is selected for activation and sent to the pipeline for scheduling. The shared memory stores the data of multiple thread blocks and the round-robin context information of each thread block.

[0007] When the thread block is reactivated, its context information is read from shared memory so that the thread block can continue to run;

[0008] The context information is a set of data related to the thread's execution state. This set of data is saved when the thread is paused or switched from an active state to an inactive state, and is reloaded before execution is resumed.

[0009] Optionally, the pipeline architecture includes: an instruction fetch module, a decoding module, a scoreboard module, a thread bundle scheduling module, a register file, an execution unit, a write-back unit, a thread block round-robin module, and a shared memory unit;

[0010] The instruction fetch module, the decoding module, the scoreboard module, and the thread bundle scheduling module are consistent with the GPGPU pipeline and use the physical thread bundle ID index when performing thread processing.

[0011] The register file is indexed using the ID of the logical thread;

[0012] The execution unit and the write-back unit are consistent with the GPGPU pipeline;

[0013] The shared storage unit contains the computation data of the thread block and the context information during the computation process, for use when the thread block is switched in turn.

[0014] Optionally, the thread block round-robin module includes a list of thread blocks and a context management controller. The list of thread blocks records the ID of the thread block, the state of the thread block, and the activity value of the thread block, respectively.

[0015] The thread block ID is the number of all thread blocks allocated to the shared memory. The thread block state includes two states: active and inactive. The activity value of the thread block is a program counter, the initial value of which is the number of thread bundles contained in the thread block.

[0016] Optionally, the context information includes: a warp number for unique identification, a thread block number for unique identification, a program counter, a SIMT stack, an execution mask and flags, and memory operation status.

[0017] Optionally, the program counter is used to indicate the current position of a thread or warp in the program.

[0018] Optionally, the SIMT stack is used to track branches and loops during thread execution and stores the address of the re-convergence point.

[0019] Optionally, the execution mask and flags are used to indicate the threads active in the warp and their state information.

[0020] Secondly, embodiments of this application also provide a live round-robin scheduling system for thread blocks, comprising:

[0021] The scheduling unit is used to receive feedback from the thread block scheduler when the thread bundle of a thread block encounters a long-delay memory access operation, match the corresponding entry of the thread block by the ID number of the feedback, decrement the activity value by 1, and when the activity value is zero, deactivate the thread block and store the execution context information of the thread block in the shared memory. At the same time, it selects the thread block with the activity value to activate and sends it to the pipeline for scheduling and execution. The shared memory stores the data of multiple thread blocks and the round-robin context information of each thread block.

[0022] The reading unit is used to read the context information of the thread block from the shared memory when the thread block is reactivated, so that the thread block can continue to run; wherein, the context information is a set of data related to the thread execution state, which is saved when the thread is paused or switched from an active state to an inactive state, and is reloaded before execution is resumed.

[0023] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described live round-robin scheduling method for thread blocks.

[0024] Fourthly, embodiments of this application also provide a storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described live round-robin scheduling method for thread blocks.

[0025] As can be seen from the above technical solutions, the present invention has the following advantages:

[0026] The thread block live round-robin scheduling method, system, electronic device, and storage medium provided in this application make full use of the shared memory and register file capacity on the SM to allocate as many thread blocks as possible to the SM until the shared memory or register file is full. It also masks the latency of long-delay instructions by polling and switching live thread blocks. Furthermore, the context management design is relatively simple, which improves the operating efficiency of the GPGPU. Attached Figure Description

[0027] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart illustrating the implementation of a live round-robin scheduling method for thread blocks provided in this embodiment of the invention;

[0029] Figure 2 A schematic diagram of a wheel cutting process provided in an embodiment of the present invention;

[0030] Figure 3 This is an improved flow diagram provided in an embodiment of the present invention;

[0031] Figure 4 This is a functional diagram of a thread block round-cutting module provided in an embodiment of the present invention;

[0032] Figure 5 This is a schematic diagram of the structure of a live round-robin scheduling system for thread blocks provided in an embodiment of the present invention;

[0033] Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0034] To better understand the technical solution of this application, the technical terms used in this application are explained as follows:

[0035] General-purpose computing on graphics processing units (GPGPUs) are graphics processors that utilize graphics processing capabilities to perform general-purpose computing tasks that would otherwise be handled by a central processing unit (CPU). These general-purpose computations are often unrelated to graphics processing. Due to the powerful parallel processing capabilities and programmable pipelines of modern GPUs, stream processors can handle non-graphics data. Especially when dealing with Single Instruction Multiple Data (SIMD) applications where the computational demands of data processing far exceed the needs of data scheduling and transfer, GPGPUs significantly outperform traditional CPU applications.

[0036] Streaming multiprocessors (SMs, also known as computing units) are one of the core components of a GPU. A GPU can have one or more SMs. SMs are considered the heart of the GPU, analogous to the core of a CPU.

[0037] A Collaborative Thread Array (CTA) is a group of threads executing on the same SM (Thread Array) that share resources such as shared memory and registers. A CTA is the basic unit of parallel execution in CUDA programs.

[0038] Various embodiments of this disclosure will be described more fully in the following detailed description. This disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of this disclosure to the specific embodiments disclosed herein, but rather this disclosure should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of this disclosure.

[0039] In the following, the terms “comprising” or “may include”, which may be used in various embodiments of this disclosure, indicate the presence of the disclosed functions or operations and do not limit the addition of one or more functions or operations. Furthermore, as used in various embodiments of this disclosure, the terms “comprising,” “having,” and their cognates are intended only to indicate a specific feature, number, step, operation, or combination of the foregoing and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, or combinations of the foregoing, or the possibility of adding one or more features, numbers, steps, operations, or combinations of the foregoing.

[0040] In various embodiments of this disclosure, the expression "or" or "at least one of A and / or B" includes any combination or all combinations of the words listed simultaneously. For example, the expression "A or B" or "at least one of A and / or B" may include A, may include B, or may include both A and B.

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0042] See Figure 1 The diagram shows a flowchart of a live round-robin scheduling method for thread blocks in a specific embodiment, including the following execution steps:

[0043] Step 100: When a thread block encounters a long-delay memory access operation, receive feedback from the thread block scheduler, match the corresponding entry of the thread block with the feedback ID number, and decrement the activity value by 1. When the activity value is zero, the thread block is deactivated, and the context information of the thread block is stored in the shared memory. At the same time, select a thread block with an activity value to activate and send it to the pipeline for scheduling and execution.

[0044] The shared storage stores data for multiple thread blocks and the switching context information for each thread block.

[0045] In some implementations, see Figure 2 As shown, the shared memory stores data for thread blocks 1 to N. It also stores the round-robin context information for each thread block, up to the upper limit of the shared memory. When the activity value of a thread block reaches 0, the thread block is deactivated, and its execution context information is stored in the shared memory. When the thread block is reactivated, the context information is read out so that the thread block can continue to run.

[0046] For example, see Figure 3 The diagram shows an improved pipeline architecture, which includes an instruction fetch module, a decode module, a scoreboard module, a thread bundle scheduling module, a register file, an execution unit, a write-back unit, a thread block round-robin module, and a shared memory unit. The instruction fetch, decode, scoreboard, and thread bundle scheduling modules are identical to the basic GPGPU pipeline and remain unchanged. During thread bundle processing, physical thread bundle IDs are used for indexing (a physical thread bundle refers to the maximum number of thread bundles that the hardware can schedule, limited by scheduling resources, including instruction buffers, scoreboards, schedulers, and SIMT stacks). The register file, to ensure it supports the maximum number of threads, uses logical thread IDs (referring to the maximum number of threads the register file can support) for indexing, because the register file stores not only data of currently active threads but also data of deactivated threads. The execution and write-back units are identical to the basic GPGPU pipeline and remain unchanged. The thread block rotation module includes a thread block state table, which records the IDs, states, and activity values ​​of all thread blocks stored in shared memory. It also includes a context management controller, which can read the information of active thread blocks from the shared memory unit during thread block rotation. The shared memory unit contains the computation data of the thread blocks and the context information during the computation process, for use during thread block rotation.

[0047] Step 101: When the thread block is reactivated, read its context information from the shared memory so that the thread block can continue to run.

[0048] The context information is a set of data related to the thread's execution state. This set of data is saved when the thread is paused or switched from an active state to an inactive state, and is reloaded before execution is resumed.

[0049] For example, the context information includes: a warp number for unique identification, a thread block number for unique identification, a program counter, a SIMT stack, an execution mask and flags, and memory operation status.

[0050] In one specific embodiment, for each thread block, the context information includes: 1. A warp number to uniquely identify the warp. 2. A thread block number to uniquely identify the thread block. 3. A program counter to indicate the current execution position of the thread or warp in the program. 4. A SIMT stack to track branches and loops during thread execution and to store the addresses of reconvergence points. 5. An execution mask and flags to indicate which threads are active in the warp and their status information, such as whether they have encountered specific execution exceptions. 6. Memory operation status, such as the status of incomplete memory operations, such as the queuing and execution status of load and store operations. The focus of context switching is to minimize the time and resources required to save and restore this information. Since register files and shared memory store virtually all thread block information, only a small portion of the status information needs to be saved and restored. This design allows for efficient context switching, reduces the time overhead caused by thread scheduling, and improves the overall efficiency of GPGPU execution.

[0051] In one specific implementation, see [reference] Figure 4 The diagram illustrates the function of the thread block round-robin module. It includes a thread block table and a context management controller. The table entries contain the thread block ID, thread block status, and thread block activity value. The thread block ID is the number of all thread blocks allocated to shared memory (e.g., 0, 1, 2). The thread block status includes active and inactive states. The activity value is a counter, initially set to the number of thread bundles contained in the thread block (e.g., 8, 0, 0). When a thread bundle in a thread block encounters a long-delay memory access operation, the thread bundle scheduler reports this to the thread block round-robin module. The module matches the corresponding thread block entry using the reported ID and decrements the activity value by 1. When the activity value reaches 0, the thread block is inactive, and its context is stored in shared memory. The module then selects the thread block with the highest activity value for activation and sends it to the pipeline for scheduling.

[0052] In this embodiment, the shared memory and register file capacity on the SM are fully utilized to allocate as many thread blocks as possible to the SM until the shared memory or register file is full. The latency of long-delay instructions is masked by polling and switching active thread blocks. The context management design is relatively simple, which improves the operating efficiency of the GPGPU.

[0053] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0054] like Figure 5 As shown, the following are embodiments of the active round-robin scheduling system for thread blocks provided in this disclosure. These systems belong to the same inventive concept as the active round-robin scheduling methods for thread blocks described in the above embodiments. For details not described in detail in the embodiments of the active round-robin scheduling system for thread blocks, please refer to the embodiments of the active round-robin scheduling methods for thread blocks described above.

[0055] The scheduling unit 50 is used to receive feedback from the thread bundle scheduler when the thread bundle of a thread block encounters a long-delay memory access operation, match the corresponding entry of the thread block by the ID number of the feedback, decrement the activity value by 1, and when the activity value is zero, deactivate the thread block and store the running context information of the thread block in the shared memory. At the same time, it selects the thread block with the activity value to activate and sends it to the pipeline for scheduling and execution. The shared memory stores the data of multiple thread blocks and the round-robin context information of each thread block.

[0056] The reading unit 51 is used to read the context information from the shared storage when the thread block is reactivated, so that the thread block can continue to run; wherein, the context information is a set of data related to the thread execution state, which is saved when the thread is paused or switched from an active state to an inactive state, and is reloaded before execution is resumed.

[0057] Figure 6 This is a schematic diagram of the hardware structure of an electronic device that implements various embodiments of the present invention.

[0058] The live round-robin scheduling method for thread blocks provided in this application can be applied to electronic devices. Those skilled in the art will understand that the electronic device structure involved in the embodiments of this invention does not constitute a limitation on the electronic device. An electronic device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, electronic devices include, but are not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0059] Electronic devices may include processors, external memory interfaces, internal memory, universal serial bus (USB) interfaces, charging management modules, power management modules, batteries, wireless communication modules, audio modules, speakers, microphones, sensor modules, buttons, cameras, displays, and SIM card interfaces, etc.

[0060] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0061] A processor may include one or more processing units, such as: a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0062] The processor can serve as the nerve center and command center of an electronic device. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0063] The processor may also include memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or that are used repeatedly. If the processor needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency.

[0064] An external storage interface (ESI) can be used to connect external memory cards, such as microSD cards, to expand the storage capacity of electronic devices. The external memory card communicates with the processor through the ESI to perform data storage functions, such as saving music and video files on the external memory card.

[0065] Internal memory can be used to store computer executable program code, which includes instructions. The processor executes various functional applications and data processing of electronic devices by running the instructions stored in internal memory. Internal memory can include a program storage area and a data storage area. Internal memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0066] Wireless communication functionality in electronic devices can be achieved through antennas, wireless communication modules, modem processors, and baseband processors.

[0067] Wireless communication modules can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies.

[0068] Electronic devices can implement audio functions through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.

[0069] Electronic devices can achieve shooting functions through ISPs, cameras, video codecs, GPUs, displays, and application processors.

[0070] Electronic devices can achieve display functions through GPUs, displays, and application processors.

[0071] A GPU is a microprocessor for image processing, connected to the display screen and application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. A processor may include one or more GPUs, which execute program instructions to generate or modify display information.

[0072] A display screen is used to display images, videos, etc. A display screen includes a display panel.

[0073] The aforementioned electronic device implements the thread block live round-robin scheduling method of this application, which stores data of multiple thread blocks and round-robin context information of each thread block in shared memory until the upper limit of shared memory is reached. When a thread block's thread bundle encounters a long-delay memory access operation, it receives feedback from the thread bundle scheduler, matches the corresponding thread block entry using the feedback ID number, and decrements the liveness value by 1. When the liveness value is zero, the thread block is deactivated, and the running context information of the thread block is stored in shared memory. Simultaneously, a thread block with a liveness value is selected for activation and sent to the pipeline for scheduling. When the thread block is reactivated... When the thread block continues to run, its context information is read from the shared memory. The context information is a set of data related to the thread's execution state. This set of data is saved when the thread is paused or switched from an active state to an inactive state, and is reloaded before execution is resumed. This technical solution makes full use of the shared memory and register file capacity on the SM, allocates as many thread blocks as possible to the SM until the shared memory or register file is full, and masks the latency of long-delay instructions by polling and switching active thread blocks. The context management design is relatively simple, which has the beneficial effect of improving the running efficiency of GPGPU.

[0074] The storage medium provided in this application stores a program product capable of implementing a live round-robin scheduling method for thread blocks.

[0075] The live round-robin scheduling method for thread blocks includes: storing data of multiple thread blocks and round-robin context information of each thread block in shared memory until the upper limit of shared memory is reached; when a thread block's thread bundle encounters a long-delay memory access operation, receiving feedback from the thread bundle scheduler, matching the corresponding thread block entry by the feedback ID number, and decrementing the liveness value by 1; when the liveness value is zero, deactivating the thread block, storing the running context information of the thread block in shared memory, and simultaneously selecting a thread block with a liveness value to activate and sending it to the pipeline for scheduling; when the thread block is reactivated, reading its context information from shared memory so that the thread block can continue to run; wherein, the context information is a set of data related to the thread execution state, which is saved when the thread is paused or switched from an active state to an inactive state, and is reloaded before resuming execution.

[0076] In some possible implementations, the subject matter of this disclosure, the method and system for active round-robin scheduling of thread blocks, can be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0077] The storage medium disclosed herein may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0078] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A live round-robin scheduling method for thread blocks, characterized in that, include: When a thread block encounters a long-delay memory access operation, it receives feedback from the thread block scheduler, matches the corresponding entry of the thread block with the feedback ID number, and decrements the activity value by 1. When the activity value is zero, the thread block is deactivated, and the execution context information of the thread block is stored in the shared memory. At the same time, a thread block with an activity value is selected for activation and sent to the pipeline for scheduling. The shared memory stores the data of multiple thread blocks and the round-robin context information of each thread block. When the thread block is reactivated, its context information is read from shared memory so that the thread block can continue to run; The context information is a set of data related to the thread's execution state. This set of data is saved when the thread is paused or switched from an active state to an inactive state, and is reloaded before execution is resumed. The pipeline architecture includes: instruction fetch module, decoding module, scoreboard module, thread bundle scheduling module, register file, execution unit, write-back unit, thread block round-robin module, and shared memory unit; The instruction fetch module, the decoding module, the scoreboard module, and the thread bundle scheduling module are consistent with the GPGPU pipeline and use the physical thread bundle ID index when performing thread processing. The register file is indexed using the ID of the logical thread; The execution unit and the write-back unit are consistent with the GPGPU pipeline; The shared storage unit contains the computation data of the thread block and the context information during the computation process, for use when the thread block is switched in turn; The thread block round-robin module includes a list of thread blocks and a context management controller. The list of thread blocks records the ID of the thread block, the state of the thread block, and the activity value of the thread block. The ID of a thread block is the number of all thread blocks allocated to the shared memory. The state of a thread block includes two states: active and inactive. The activity value of a thread block is a program counter, the initial value of which is the number of thread bundles contained in the thread block. The context information includes: a warp number, a thread block number, a program counter, a SIMT stack, an execution mask and flags, and memory operation status; The program counter is used to indicate the current execution position of a thread or warp within the program; The SIMT stack is used to track branches and loops during thread execution and stores the addresses of re-convergence points; When processing thread bundles, physical thread bundle IDs are used for indexing. A physical thread bundle refers to the maximum number of thread bundles that the hardware can schedule, which is limited by scheduling resources, including instruction buffers, scoreboards, schedulers, and SIMT stacks. The register file, to ensure it supports the maximum number of threads, uses logical thread IDs for indexing. This is because the register file stores not only data of currently active threads but also data of deactivated threads; therefore, logical thread IDs are required for indexing. When a thread block encounters a long-delay memory access operation, the thread block scheduler will report back to the thread block round-robin module. The thread block round-robin module will match the corresponding entry of the thread block with the reported ID number and decrement the activity value by 1. When the activity value is 0, the thread block will be deactivated and the corresponding context will be stored in the shared memory. The module will then select the thread block with the higher activity value to activate and send it to the pipeline for scheduling and execution.

2. The live round-robin scheduling method for thread blocks according to claim 1, characterized in that, The execution mask and flags are used to indicate the threads active in the warp and their status information.

3. A live round-robin scheduling system for thread blocks applied to the live round-robin scheduling method for thread blocks according to any one of claims 1-2, characterized in that, include: The scheduling unit is used to receive feedback from the thread block scheduler when the thread bundle of a thread block encounters a long-delay memory access operation, match the corresponding entry of the thread block by the ID number of the feedback, decrement the activity value by 1, and when the activity value is zero, deactivate the thread block and store the execution context information of the thread block in the shared memory. At the same time, it selects the thread block with the activity value to activate and sends it to the pipeline for scheduling and execution. The shared memory stores the data of multiple thread blocks and the round-robin context information of each thread block. The reading unit is used to read the context information of the thread block from the shared memory when the thread block is reactivated, so that the thread block can continue to run; wherein, the context information is a set of data related to the thread execution state, which is saved when the thread is paused or switched from an active state to an inactive state, and is reloaded before execution is resumed.

4. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the live round-robin scheduling method for thread blocks as described in any one of claims 1 to 2.

5. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the live round-robin scheduling method for thread blocks as described in any one of claims 1 to 2.

Images