Task scheduling method, system, chip and program product

Abstract

This application provides a task scheduling method, system, chip, and program product, relating to the field of chip technology. The task scheduling method includes: a local scheduler notifying a computing unit group to initiate a query request to a global scheduler; the global scheduler obtaining quantity information based on the query request, the quantity information indicating the number of available computing units in the computing unit group; the global scheduler determining a first thread block group based on the quantity information; the global scheduler issuing the first thread block group to the computing unit group; and the local scheduler allocating the thread blocks in the first thread block group to available computing units in the computing unit group for execution. This method improves the utilization rate of the system's computing resources.

Description

Technical Field

[0001] This application relates to the field of chip technology, and in particular to a task scheduling method, system, chip, and program product. Background Technology

[0002] With the development of chip technology, the shared memory of multiple computing units can be interconnected to form computing unit groups. Multiple thread blocks within a computing unit group can share data in shared memory and collaboratively execute computing tasks to improve computing efficiency.

[0003] In related technologies, the scheduler distributes thread block groups to computing unit groups, and each thread block group includes multiple thread blocks. After completing the current computing task, the computing unit group sends a query request to the scheduler. In response to the query request, the scheduler allocates a new thread block group to the computing unit group to continue executing the next round of computing tasks.

[0004] However, due to the difference in the actual number of available computing units in different computing unit groups within the system, some computing unit groups may be unable to continuously execute computing tasks due to insufficient available computing units, thereby causing a decrease in the utilization rate of the system's computing resources. Summary of the Invention

[0005] This application provides a task scheduling method, system, chip, and program product. The technical solutions provided by this application are as follows.

[0006] According to one aspect of the embodiments of this application, a task scheduling method is provided, applied to a global scheduler, the method comprising: Receive a query request from a computing unit group, the computing unit group including at least one computing unit; Based on the query request, quantity information is obtained, which indicates the number of available computing units in the computing unit group; Based on the quantity information, a first thread block group is determined, wherein the number of thread blocks in the first thread block group matches the quantity information; The first thread block group is issued to the computing unit group.

[0007] According to one aspect of the embodiments of this application, a task scheduling method is provided, applied to a local scheduler, the local scheduler corresponding to a group of computing units, the group of computing units including at least one computing unit, the method comprising: The computing unit group is notified to initiate a query request to the global scheduler. The query request is used to trigger the global scheduler to issue a first thread block group to the computing unit group. The number of thread blocks in the first thread block group matches the number of available computing units in the computing unit group. The thread blocks in the first thread block group are assigned to the available computing units in the computing unit group for execution.

[0008] According to one aspect of the embodiments of this application, a task scheduling system is provided, the system including a global scheduler and at least one local scheduler; wherein, the local scheduler corresponds to a computing unit group, and the computing unit group includes at least one computing unit; The local scheduler is used to notify the computing unit group to initiate a query request to the global scheduler; The global scheduler is configured to obtain quantity information based on the query request, the quantity information indicating the number of available computing units in the computing unit group; determine a first thread block group based on the quantity information, the number of thread blocks in the first thread block group matching the quantity information; and issue the first thread block group to the computing unit group. The local scheduler is also configured to allocate thread blocks in the first thread block group to available computing units in the computing unit group for execution.

[0009] According to one aspect of the embodiments of this application, a global scheduler is provided, the global scheduler comprising: A receiving unit is configured to receive a query request from a group of computing units, the group of computing units including at least one computing unit. The processing unit is configured to obtain quantity information based on the query request, the quantity information indicating the number of available computing units in the computing unit group; and determine a first thread block group based on the quantity information, wherein the number of thread blocks in the first thread block group matches the quantity information. The sending unit is configured to send the first thread block group to the computing unit group.

[0010] According to one aspect of the embodiments of this application, a local scheduler is provided, the local scheduler corresponding to a computing unit group, the computing unit group including at least one computing unit, the local scheduler comprising: The sending unit is configured to notify the computing unit group to initiate a query request to the global scheduler to instruct the global scheduler to issue a first thread block group to the computing unit group, wherein the number of thread blocks in the first thread block group matches the number of available computing units in the computing unit group; The processing unit is configured to allocate thread blocks from the first thread block group to available computing units in the computing unit group for execution.

[0011] According to one aspect of the embodiments of this application, a computer device is provided, the computer device including a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above-described task scheduling method.

[0012] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program is loaded and executed by a processor to implement the above-described task scheduling method.

[0013] According to one aspect of the embodiments of this application, a chip is provided, the chip including programmable logic circuits and / or program instructions, which, when the chip is running, are used to implement the above-described task scheduling method.

[0014] According to one aspect of the embodiments of this application, a computer program product is provided, the computer program product including a computer program, the computer program being loaded and executed by a processor to implement the above-described task scheduling method.

[0015] The technical solution provided in this application can bring the following beneficial effects: The local scheduler instructs the computing unit group to initiate a query request to the global scheduler. Based on the query request, the global scheduler obtains information on the number of available computing units in the computing unit group, determines the first thread block group based on this information, and distributes the first thread block group to the computing unit group. The local scheduler then allocates the thread blocks in the first thread block group to the available computing units in the computing unit group for execution. Therefore, even when the number of available computing units in a computing unit group is small, the global scheduler can still distribute suitable thread block groups, enabling the computing unit group to execute its computing tasks normally. In summary, the technical solution provided by this application embodiment enables computing unit groups with a small number of available computing units to execute computing tasks normally, thereby improving the utilization rate of the system's computing resources. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of a task scheduling system provided in one possible implementation of this application; Figure 2 This is a flowchart of a task scheduling method provided in one possible implementation of this application; Figure 3 This is a schematic diagram of a local scheduler and computing unit group provided in one possible implementation of this application; Figure 4 This is a schematic diagram of a global scheduler provided in one possible implementation of this application; Figure 5 This is a schematic diagram of the task scheduling process provided in one possible implementation of this application; Figure 6 This is a block diagram of the global scheduler provided in one possible implementation of this application; Figure 7 This is a block diagram of the local scheduler provided in one possible implementation of this application; Figure 8 This is a structural block diagram of a computer device provided in one possible implementation of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0019] Before introducing the technical solution proposed in this application, the relevant technical background and related technologies will be briefly described below.

[0020] In the field of chip technology, processors typically employ a multi-level thread organization architecture to manage a large number of computing units when performing massively parallel computing tasks. Taking a graphics processing unit (GPU) as an example, its kernel can adopt a three-level structure of grid, workgroup, and thread. A grid corresponds to a complete kernel call and contains several workgroups; each workgroup contains dozens to thousands of threads, and threads within a workgroup can perform fast data exchange and synchronization through shared memory.

[0021] With the increasing demand for AI (Artificial Intelligence) computing, processor hardware designs are trending towards interconnecting the shared memory of multiple computing units to form computing unit clusters. For example, in GPUs, the shared memory of multiple multi-threaded processors (MPs) can be interconnected to form a multi-threaded processor cluster (MPC). Multiple thread blocks within a computing unit cluster can share data in shared memory and collaboratively execute computational tasks to improve computational efficiency. Building upon this, the concept of thread block groups has been introduced, which consist of multiple thread blocks, their size limited by the number of computing units within a single computing unit cluster.

[0022] For task scheduling in computing unit group scenarios, the industry has proposed a variety of load balancing methods.

[0023] One approach is hardware load balancing. Users divide the overall computing task into multiple task blocks, with each thread block or group of thread blocks executing one task block. The front-end scheduler distributes thread blocks to computing units with available resources based on the load of each computing unit. When scheduling is done at the thread block group level, all computing units required by that thread block group must be idle before the scheduler distributes the thread block group to the corresponding computing unit group. However, this method may lead to some computing unit groups waiting for extended periods due to individual computing units being busy, resulting in decreased resource utilization, especially when the number of computing units is inconsistent.

[0024] Another approach is software load balancing. Users set the grid size to a multiple of the number of computing units, and the scheduler issues thread blocks only once or a few times. Each thread block continuously processes multiple task blocks in a loop, processing one task block at a time. The thread block can also atomically retrieve new tasks from the global task pool. However, because the total number of tasks processed by a thread block is determined at the initial issuance, and each computing unit may be executing other tasks at the time of issuance, the load balancing effect is not ideal.

[0025] In related technologies, a hardware-software combined load balancing mechanism has been proposed. In this mechanism, the user divides the overall computing task into multiple task blocks, and each thread block group executes one task block at a time. During the task startup phase, the scheduler, based on the load status of each computing unit group, distributes thread block groups to computing unit groups with available resources. When the available resources on a computing unit group are insufficient to receive new thread block groups, the scheduler pauses distribution. Unlike hardware load balancing, after completing the currently allocated thread block group, the computing unit group does not immediately release its occupied computing resources and end execution. Instead, the program running on the computing unit group initiates a query request to the scheduler. Upon receiving the query request, the scheduler retrieves a new thread block group from its internally maintained scheduling queue, returns it to the requesting computing unit group, and removes the task record from the scheduling queue. Subsequently, the computing unit group continues to execute the next round of computing tasks based on the newly obtained thread block group. This process is repeated until there are no tasks to be allocated in the scheduling queue, at which point the scheduler returns a query failure signal, and the computing unit group exits execution.

[0026] However, due to factors such as manufacturing processes, the actual number of available computing units within different computing unit groups in the system may vary. Some computing unit groups may have fewer available computing units, and when the scheduler allocates new thread block groups, it cannot perceive the actual available resources of each computing unit group. The size of the allocated thread block group may not match the available resources of the computing unit group. When the number of computing units required by the allocated thread block group exceeds the number of available computing units in the computing unit group, the computing unit group cannot support the allocated thread block group, thus failing to continuously execute computing tasks. This results in the computing resources in the computing unit group being idle, leading to a decrease in the system's computing resource utilization.

[0027] To address the aforementioned issues, this application provides a task scheduling method.

[0028] First, before describing the task scheduling method provided in this application, a task scheduling system proposed in this application will be briefly described. The task scheduling system includes a global scheduler and at least one local scheduler. The local scheduler corresponds to a group of computing units, and the group of computing units includes at least one computing unit.

[0029] In one aspect of this embodiment, the global scheduler is a unit responsible for task scheduling. The global scheduler receives query requests from the computing unit group and distributes thread block groups to the computing unit group according to the query requests. Exemplarily, the global scheduler can be implemented as a front-end scheduling unit in the processor, as a processor-independent scheduling controller, or as a scheduling software module running on the processor. This application embodiment does not limit the specific implementation form of the global scheduler.

[0030] In one aspect of this embodiment, the task scheduling system further includes at least one group of computing units, each group comprising at least one computing unit. A computing unit is a basic hardware unit that performs computing tasks. For example, in a GPU, a computing unit may be a stream processor; in a Central Processing Unit (CPU), a computing unit may be a core; and in a Digital Signal Processor (DSP), a computing unit may be an execution unit. This application does not limit the specific type of computing unit.

[0031] In one aspect of this embodiment, a computing unit group is a collection formed by aggregating multiple computing units. Multiple computing units within a computing unit group can be interconnected through shared memory, enabling multiple thread blocks located within the same computing unit group to share data and collaborate synchronously. For example, in a GPU, a computing unit group can be a stream processor cluster, where multiple stream processors are interconnected via shared memory to form a stream processor cluster; in a CPU, a computing unit group can be a core cluster, where multiple processing cores are interconnected via shared cache to form a core cluster; in a DSP, a computing unit group can be an execution unit group. This application does not limit the specific organizational form of the computing unit group.

[0032] In one aspect of this embodiment, the local scheduler is a scheduling unit corresponding to a computing unit group. The local scheduler is responsible for managing and notifying the computing unit groups to initiate query requests to the global scheduler. It is worth noting that the correspondence between the local scheduler and the computing unit group can be implemented in various ways. For example, there can be a one-to-one correspondence between the local scheduler and the computing unit group, with each computing unit group configured with an independent local scheduler; another example is that one local scheduler corresponds to multiple computing unit groups, uniformly managing the synchronization state of multiple computing unit groups; yet another example is that the local scheduler is integrated into the hardware circuitry of the computing unit group as a functional module within the computing unit group. This application does not limit the correspondence between the local scheduler and the computing unit group, nor does it specify the exact location of the local scheduler; these details will not be elaborated upon below.

[0033] In some embodiments, the local scheduler and the global scheduler can interact via a Network on Chip (NoC) or other communication interconnection structures. For example, a query request initiated by a compute unit group and notified by the local scheduler is transmitted to the global scheduler via the Network on Chip; similarly, a thread block group issued by the global scheduler is transmitted to the corresponding compute unit group via the Network on Chip.

[0034] In some embodiments, exemplarily, please refer to Figure 1 , Figure 1 This is a schematic diagram of a task scheduling system provided in one possible implementation of this application. The task scheduling system includes a global scheduler 110, local schedulers 120, computing unit groups 130, an on-chip network 140, and a grid 150. Each local scheduler 120 corresponds to a computing unit group 130, and each computing unit group 130 includes multiple computing units. The grid 150 corresponds to a complete task invocation and contains several thread block groups. The global scheduler 110 communicates with each computing unit group 130 through the on-chip network 140, and is responsible for distributing thread block groups to the computing unit groups 130 and receiving query requests from the computing unit groups 130.

[0035] Figure 1 Only one local scheduler 120 is shown, which corresponds to one computation unit group 130. The local scheduler 120 is used to allocate thread blocks from the thread block group to available computation units in the computation unit group 130 for execution. It is worth noting that each computation unit group 130 corresponds to a local scheduler 120. Figure 1 Only one local scheduler 120 is schematically shown in the figure; the local schedulers 120 corresponding to the other computing unit groups 130 are not shown in the figure.

[0036] It should be emphasized that the correspondence between the local scheduler 120 and the computing unit group 130 can be implemented in various ways. For example, the local scheduler 120 and the computing unit group 130 can correspond one-to-one; another example is that one local scheduler 120 corresponds to multiple computing unit groups 130; yet another example is that the local scheduler 120 is a functional module within the computing unit group 130. Figure 1 The computing unit group 130 shown is represented by a dashed box to indicate that the relative position between the local scheduler 120 and the computing unit group 130 is not limited; that is, the local scheduler 120 can be integrated inside the computing unit group 130 or can be independent of the computing unit group 130. This application embodiment does not limit the specific correspondence between the local scheduler 120 and the computing unit group 130, and will not elaborate further below.

[0037] It is worth noting that, Figure 1 These are merely illustrative examples. Any system architecture that can implement the task scheduling method provided in the embodiments of this application falls within the protection scope of this application.

[0038] Next, the task scheduling method provided in this application will be explained.

[0039] For example, please refer to Figure 2 , Figure 2This is a flowchart of a task scheduling method provided in one possible implementation of this application. The task scheduling method may include at least one of the following steps 210-250. Steps 210 and 250 are applied to the local scheduler side, and steps 220-240 are applied to the global scheduler side.

[0040] Step 210: The local scheduler notifies the computing unit group to send a query request to the global scheduler.

[0041] In one aspect of this embodiment, the query request is a request initiated by the computing unit group to the global scheduler to request a new thread block group. The query request triggers the global scheduler to issue a new thread block group to the computing unit group. For example, the query request may include identification information of the computing unit group so that the global scheduler can identify the computing unit group that initiated the request; or, for example, the query request may include available resource information of the computing unit group so that the global scheduler can issue a matching thread block group based on available resources. This embodiment does not limit the specific content of the query request.

[0042] In one aspect of this embodiment, the local scheduler can notify the computing unit group in various ways. For example, the local scheduler can broadcast a synchronization completion signal to its corresponding computing unit group. After receiving the synchronization completion signal, each computing unit in the computing unit group will have one of its computing units (or a thread bundle running on that computing unit) initiate a query request to the global scheduler on behalf of the entire computing unit group. Alternatively, the local scheduler can directly send a notification signal to a pre-designated representative computing unit in the computing unit group, which will then initiate a query request. Yet another example is that the local scheduler can directly generate a query request and send it to the global scheduler via the on-chip network after confirming synchronization completion, without needing to go through the computing unit group. This application embodiment does not limit the specific method by which the local scheduler notifies the computing unit group.

[0043] In some embodiments, query requests are transmitted to the global scheduler via an on-chip network or other communication interconnection structure. The computing unit group and the global scheduler may have a dedicated communication channel, or they may communicate via a shared bus or network. This application embodiment does not limit the transmission path and transmission protocol of the query request.

[0044] In some embodiments, the local scheduler can notify the computation unit group to initiate a query request after the thread block group synchronization is complete, enabling the computation unit group to continuously acquire new thread block groups and thus continuously participate in the computation task. For example, after completing the current computation task, the computation unit group does not release the computational resources it occupies, but instead initiates a query request to acquire new thread block groups and continue execution until the global scheduler returns a request failure signal, at which point the computation unit group exits execution. Through this mechanism, the computation unit group achieves persistent occupation of computational resources and continuous task execution, reducing the overhead caused by frequent allocation and release of computational resources.

[0045] It is worth noting that the above-mentioned method of the local scheduler notifying the computing unit group to initiate a query request is only an illustrative example. The embodiments of this application do not limit the specific generation conditions, notification methods and transmission methods of the query request.

[0046] Step 220: The global scheduler obtains quantity information based on the query request. The quantity information is used to indicate the number of available computing units in the computing unit group.

[0047] In one aspect of this embodiment, the quantity information is information used to indicate the number of currently available computing units in the computing unit group. Exemplarily, the quantity information can be represented in various forms. For example, the quantity information can be a specific numerical value, directly representing the number of available computing units; or it can be a ratio value, representing the proportion of available computing units to the total number of computing units in the computing unit group; or it can be a list of the available status of each computing unit in the computing unit group. This application embodiment does not limit the specific form of the quantity information.

[0048] In one aspect of this embodiment, the global scheduler obtains quantity information based on the query request. It is worth noting that the global scheduler can obtain quantity information in various ways. For example, it may include, but is not limited to: 1. In some embodiments, the query request directly carries quantity information. When a computing unit group initiates a query request, it encapsulates the quantity information of available computing units in the query request, and the global scheduler can directly parse the quantity information from the query request.

[0049] 2. In some embodiments, the query request carries identification information of the computing unit group. The global scheduler retrieves the quantity information from a preset storage area based on this identification information. For example, the local scheduler and / or computing units maintain the availability status of each computing unit in the computing unit group in their storage units. The global scheduler can access these storage units to retrieve the corresponding quantity information based on the identification information. Alternatively, the availability status information of the computing unit group can be centrally stored in a shared storage area accessible to the global scheduler, and the global scheduler reads the quantity information from this shared storage area based on the identification information.

[0050] 3. In some embodiments, the query request carries the availability status information of each computing unit in the computing unit group. The global scheduler obtains the number of available computing units by performing statistical calculations on this availability status information. For example, the query request may carry mask information, where each bit of the mask information corresponds to a computing unit in the computing unit group. A set bit indicates that the corresponding computing unit is available. The global scheduler obtains the number of available computing units by counting the number of set bits in the mask information.

[0051] It is worth noting that the above-described methods for obtaining quantity information are merely illustrative examples. This application does not limit the specific method by which the global scheduler obtains quantity information, as long as the global scheduler can obtain the quantity information used to indicate the number of available computing units in the computing unit group based on the query request.

[0052] Step 230: The global scheduler determines the first thread block group based on the quantity information, and the number of thread blocks in the first thread block group matches the quantity information.

[0053] In one aspect of this embodiment, the first thread block group refers to the thread block group determined and issued by the global scheduler in response to a query request. Matching the number of thread blocks in the first thread block group with the quantity information means that the number of thread blocks contained in the first thread block group and the number of available computing units indicated by the quantity information satisfy a preset adaptation relationship. Since each thread block in the thread block group needs to be allocated to an available computing unit for execution, the number of thread blocks in the first thread block group needs to be adapted to the number of available computing units in the computing unit group to ensure that the computing unit group can support the first thread block group.

[0054] In one aspect of this embodiment, the matching preset adaptation relationship can have various implementation forms. For example, the adaptation relationship can be that the number of thread blocks in the first thread block group is equal to the number of available computing units in the computing unit group, meaning the task size and available resource size are precisely matched, and each available computing unit executes exactly one thread block. Alternatively, the adaptation relationship can be that the number of thread blocks in the first thread block group is less than the number of available computing units in the computing unit group, meaning the task size is less than the available resource size, and some available computing units may be idle, but the computing unit group can still support the thread block group. Yet another example is that the adaptation relationship can be that the number of thread blocks in the first thread block group is less than or equal to the number of available computing units, to cover both of the above situations. This application embodiment does not limit the specific adaptation relationship.

[0055] In one aspect of this embodiment, the global scheduler can determine the first thread block group through various methods based on the quantity information. Specifically, this may include, but is not limited to: 1. In some embodiments, the global scheduler maintains a global scheduling queue, which stores thread block groups to be scheduled. The global scheduler can select a thread block group from the global scheduling queue as the first thread block group based on quantity information, wherein the number of thread blocks in the selected thread block group matches the number of available computing units in the computing unit group. For example, the global scheduler can select a thread block group from the global scheduling queue whose number of thread blocks equals the number of available computing units; when no exactly matching thread block group exists, a thread block group whose number of thread blocks is less than the number of available computing units can be selected.

[0056] 2. In some embodiments, the global scheduler can split or merge existing thread block groups based on quantity information to obtain a matching first thread block group. For example, when there is a thread block group in the global scheduling queue with a number of thread blocks greater than the number of available computing units, the global scheduler can split the thread block group to obtain a first thread block group with a number of thread blocks matching the number of available computing units. As another example, when there are multiple thread block groups in the global scheduling queue with a number of thread blocks less than the number of available computing units, the global scheduler can merge these multiple thread block groups to obtain a first thread block group with a number of thread blocks matching the number of available computing units.

[0057] 3. In some embodiments, the global scheduler can directly divide a group of thread blocks from the grid whose number of thread blocks matches the number of available computing units as the first thread block group based on the quantity information, without relying on pre-stored thread block groups.

[0058] It is worth noting that the above-described method for determining the first thread block group is merely an illustrative example, and the embodiments of this application do not limit the specific method by which the global scheduler determines the first thread block group.

[0059] Step 240: The global scheduler issues the first thread block group to the computing unit group.

[0060] In one aspect of this embodiment, the global scheduler can issue the first thread block group in various ways. For example, the global scheduler can directly issue the first thread block group to the computing unit group or the local scheduler corresponding to the computing unit group via an on-chip network. The issued description information may include the coordinates of the first thread block in the first thread block group and the number of thread blocks, or it may include the complete coordinate information of each thread block in the first thread block group; this embodiment does not limit this. As an example, the global scheduler can directly issue the first thread block group to the computing unit group. After receiving the first thread block group, each available computing unit in the computing unit group can obtain its corresponding thread block and execute it according to the aforementioned description information. As another example, the global scheduler can first issue the first thread block group to the local scheduler corresponding to the computing unit group, and the local scheduler can further distribute the first thread block group to each available computing unit in the computing unit group in units of thread blocks. This embodiment does not limit the specific object or method by which the global scheduler issues the first thread block group.

[0061] Step 250: The local scheduler allocates the thread blocks in the first thread block group to the available computing units in the computing unit group for execution.

[0062] In one aspect of this embodiment, after receiving the first thread block group issued by the global scheduler, the local scheduler is responsible for specifically allocating the thread blocks in the first thread block group to the available computing units in their corresponding computing unit groups.

[0063] For example, the local scheduler can determine the distribution of available computing units in the computing unit group based on the mask information corresponding to the computing unit group. Each bit in the mask information corresponds to a computing unit, and a set bit indicates that the corresponding computing unit is available. Accordingly, the local scheduler sequentially distributes each thread block in the first thread block group to the available computing units indicated by the mask information.

[0064] As an example allocation method, the local scheduler can maintain an allocation pointer. First, it determines the list of available computing units based on the mask information; then, it maps the thread blocks in the first thread block group to the computing units in the list of available computing units in a preset order (such as ascending thread block index); finally, it sends the task information of each thread block to the corresponding computing unit through point-to-point transmission or broadcast. As another example allocation method, the local scheduler can also broadcast the first thread block group and the mask information to all computing units in the computing unit group. After receiving the above information, each computing unit determines whether it should acquire a thread block from the thread block group based on its own unit number and mask information. For example, a computing unit can calculate the index of the thread block to be executed based on the sorting position of its corresponding bit in the mask information, thereby actively acquiring the corresponding thread block. After acquiring the allocated thread block, each available computing unit in the computing unit group begins to execute the corresponding computing task.

[0065] It is worth noting that the allocation method of the local scheduler described above is only an illustrative example. The embodiments of this application do not limit the specific objects or methods by which the local scheduler allocates thread blocks to computing units.

[0066] The task scheduling method provided in this application embodiment involves a local scheduler instructing a computing unit group to initiate a query request to the global scheduler. Based on the query request, the global scheduler obtains information on the number of available computing units within the computing unit group, determines a first thread block group based on this information, and distributes the first thread block group to the computing unit group. The local scheduler then allocates the thread blocks in the first thread block group to the available computing units within the computing unit group for execution. Therefore, even when the number of available computing units in a computing unit group is small, the global scheduler can still distribute suitable thread block groups, enabling the computing unit group to execute its computing tasks normally. In summary, the task scheduling scheme provided in this application embodiment allows computing unit groups with a small number of available computing units to execute computing tasks normally, thereby improving the utilization rate of the system's computing resources.

[0067] In some embodiments, the method further includes: receiving a synchronization request from at least one thread bundle in a first thread block group, the synchronization request being sent by a computing unit of the assigned thread bundle after completing the corresponding computing task; obtaining the synchronization completion status of the first thread block group according to the synchronization request; and, if the synchronization completion status is that synchronization has been completed, notifying the computing unit group to initiate the next query request to the global scheduler.

[0068] In this embodiment, to accurately determine whether a thread block group has completed synchronization, it is necessary to clarify the nesting and synchronization relationships among thread block groups, thread blocks, and thread bundles. For example, a thread block group contains several thread blocks, each thread block contains several thread bundles, and each thread bundle contains a fixed number of threads (e.g., 32). A thread bundle is the basic unit of hardware scheduling; threads within the same thread bundle execute synchronously in Single Instruction Multiple Thread (SIMT) mode. Therefore, the completion status of a thread bundle can reflect part of the progress of its corresponding thread block. Once all thread bundles within a thread block have been executed, the thread block is considered synchronized. Similarly, once all thread blocks within a thread block group have been executed, the thread block group is considered synchronized. Based on these relationships, the local scheduler can deduce the synchronization completion status of thread blocks and thread block groups level by level based on the completion status of each thread bundle (i.e., receiving synchronization requests at the thread bundle granularity). When a thread block group has completed synchronization, the scheduler notifies the computation unit group to initiate the next query request to the global scheduler.

[0069] It is worth noting that in other processor architectures such as CPUs or DSPs, a thread bundle can correspond to the smallest processing unit capable of independently executing synchronization instructions. This application uses a thread bundle as an example for illustration, and does not limit the specific implementation of the smallest processing unit. Furthermore, there are multiple implementation methods for sending synchronization requests. For example, a thread bundle can trigger the sending of a synchronization request by executing a synchronization instruction; or, a thread bundle can automatically trigger the sending of a synchronization request when exiting execution. This application does not limit the specific sending method and triggering conditions of the synchronization request.

[0070] In one aspect of this embodiment, the local scheduler receives and processes thread bundle-level synchronization requests to monitor the execution progress of the first thread block group in real time. Exemplarily, the above process may include, but is not limited to, the following steps: 1. Receiving a Synchronization Request: When a computing unit in a computing unit group completes the computing task of its assigned thread bundle, the computing unit sends a synchronization request to the local scheduler. For example, the synchronization request can be triggered by the thread bundle executing a dedicated synchronization instruction, or it can be triggered automatically when the task is completed and ready to exit.

[0071] 2. Obtaining Synchronization Completion Status: The local scheduler determines the number of thread bundles that have completed synchronization based on the received synchronization requests. When the local scheduler determines that all thread bundles requiring synchronization in the first thread block group have sent synchronization requests, it considers the first thread block group to have completed synchronization.

[0072] 3. Notification to initiate the next query: After confirming the synchronization of the first thread block group, the local scheduler notifies its corresponding computation unit group to initiate the next query request to the global scheduler to obtain the new thread block group and continue executing the computation task. This mechanism ensures that the computation unit group can initiate a query request promptly after the current computation task is completed, avoiding idle waiting of computational resources.

[0073] By employing the mechanism described above—obtaining synchronization completion status based on synchronization requests and notifying the initiation of the next query request upon synchronization completion—the local scheduler can accurately determine the synchronization completion status of thread block groups based on the actual execution status of each thread bundle. Compared to directly initiating query requests without a synchronization determination mechanism, this embodiment ensures that query requests are only triggered after the current thread block group has completed synchronization, avoiding inaccurate information on the number of available computing units due to premature query requests; it also avoids the problem of idle computing resources due to delayed query requests. Therefore, this embodiment enables computing unit groups to continuously obtain matching thread block groups, thereby further improving the system's task scheduling efficiency and computing resource utilization.

[0074] In some embodiments, before receiving a synchronization request from at least one thread bundle in the first thread block group, the method further includes: allocating a table entry for the first thread block group and setting counter information in the table entry, the counter information being used to record the synchronization state of the first thread block group.

[0075] Based on the synchronization request, obtain the synchronization completion status of the first thread block group, including: updating the count information according to the synchronization request; and obtaining the synchronization completion status of the first thread block group based on the count information.

[0076] In one aspect of this embodiment, the local scheduler includes a storage unit for storing entries for thread block groups. Exemplarily, this storage unit can be a register set, static random access memory (SRAM), dynamic random access memory (DRAM), or any other type of storage unit. For example, in a GPU, this storage unit can be a workgroup cluster buffer (WGC buffer), which can be located on the local scheduler side and is specifically used to manage the synchronization state information of thread block groups. This application embodiment does not limit the specific implementation form and physical location of the storage unit.

[0077] In some embodiments, an entry is a storage entry allocated to a thread block group in a storage unit. For each thread block group issued to a computation unit group, the local scheduler allocates a corresponding entry in its storage unit. This entry stores count information used to track the synchronization status of the thread block group. Further, the entry can be identified by the thread block group index number (workgroup_cluster_id), which serves as the identification information for the thread block group and is issued along with the thread block group to each computation unit in the computation unit group. The computation unit can carry this identification information when subsequently sending synchronization requests for thread bundles, allowing the local scheduler to locate the corresponding entry based on the identification information.

[0078] In one aspect of this embodiment, the local scheduler receives synchronization requests from each thread bundle in the thread block group, updates the count information in the table entries according to the synchronization requests, and obtains the synchronization completion status of the first thread block group. For example, the local scheduler determines the number of thread bundles that have sent synchronization requests based on the count information. When the count information indicates that all thread bundles requiring synchronization in the thread block group have completed synchronization, the local scheduler determines that the thread block group synchronization is complete.

[0079] It is worth noting that the specific implementation of the above-mentioned storage unit and table entries, the method of sending synchronization requests, and the method of updating counting information are merely illustrative examples, and the embodiments of this application do not limit these aspects.

[0080] By allocating entries and setting counter information for the first thread block group, and updating the counter information according to synchronization requests and obtaining the synchronization completion status based on the counter information, the local scheduler can independently and accurately track and manage the synchronization status of each thread block group on an entry-by-entry basis. Since each entry corresponds one-to-one with a thread block group, and the counter information dynamically reflects the synchronization progress of each thread bundle, the local scheduler can accurately determine at any time whether the current thread block group has completed synchronization, thus triggering the next query request at the appropriate time when synchronization is complete.

[0081] In some embodiments, the counting information includes an expected count and an arrival count. The expected count indicates the number of thread bundles in the first thread block group that need to complete synchronization, and the arrival count indicates the number of thread bundles in the first thread block group that have already completed synchronization. Initializing the counting information in the table entry includes setting the expected count and arrival count to the total number of thread bundles contained in the first thread block group.

[0082] In one aspect of this embodiment, the expected count represents the total number of thread bundles in the first thread block group that need to be synchronized within the current synchronization period. For example, the expected count can be set to the total number of thread bundles contained in the first thread block group. When a thread bundle exits its computational task, the expected count decreases accordingly to dynamically reflect the number of thread bundles currently capable of executing computational tasks. The arrival count represents the number of thread bundles in the thread block group that have sent synchronization requests within the current synchronization period. The arrival count is updated accordingly each time a synchronization request from a thread bundle is received. Using the expected count and the arrival count, the local scheduler can accurately determine whether all thread bundles in the first thread block group have completed synchronization and / or have exited their computational tasks.

[0083] In one aspect of this embodiment, the total number of thread bundles refers to the sum of the number of thread bundles contained in all thread blocks within the first thread block group. For example, assuming the first thread block group contains 4 thread blocks, and each thread block contains 8 thread bundles (each thread bundle contains 32 threads), then the total number of thread bundles contained in the first thread block group is 32, meaning both the expected count and the arrival count are set to 32. This embodiment does not limit how the total number of thread bundles is specifically calculated from the number of thread blocks and the number of thread bundles per thread block.

[0084] In some embodiments, the timing of setting the expected count and arrival count can coincide with the timing of entry allocation; that is, when the local scheduler allocates entries for a thread block group, the setting operations for the expected count and arrival count are completed synchronously. Specifically, after receiving the first thread block group from the global scheduler, the local scheduler triggers the entry allocation and count setting process. The local scheduler first requests a free entry in its storage unit and obtains the index number of that entry as the identification information for the first thread block group. Then, the local scheduler calculates the total number of thread bundles in the first thread block group based on the number of thread blocks contained in the first thread block group and the number of thread bundles contained in each thread block. Finally, the local scheduler writes the calculated total number of thread bundles into the expected count and arrival count, respectively, completing the setting operation.

[0085] In some embodiments, the expected count and the arrival count may also be different. For example, when some thread bundles in the first thread block group have exited the execution of the computation task before the entry is assigned, the expected count may be set to the number of thread bundles that have not yet exited the execution, while the arrival count is initialized to zero.

[0086] It is worth noting that the specific composition and setting method of the above-mentioned counting information are only illustrative examples, and the embodiments of this application do not limit the specific form of the counting information, the set value, or the setting time.

[0087] By setting the expected count and the arrival count, and initializing their values ​​to the total number of thread bundles, a clear initial benchmark is provided for tracking the synchronization state, making the judgment of synchronization completion deterministic, thereby ensuring the reliability of the timing of query request initiation.

[0088] In some embodiments, the counting information further includes a survival count, which indicates the number of thread blocks in the first thread block group that have not exited execution. Setting the counting information in the table entry includes setting the survival count to the total number of thread blocks in the first thread block group.

[0089] In one aspect of this embodiment, the liveness count is a count used to determine the liveness status of thread blocks in the first thread block group. Unlike the expectation count and arrival count, which indicate the synchronization status at the thread bundle level, the liveness count indicates the exit status at the thread block level. A thread block group contains multiple thread blocks, and each thread block contains multiple thread bundles. When all thread bundles within a thread block exit to perform computational tasks, the entire thread block exits, and the liveness count is decremented by one. By managing the liveness status of thread blocks separately from the synchronization status of thread bundles, the local scheduler can accurately grasp the overall state of the first thread block group.

[0090] In one aspect of this embodiment, it is necessary to distinguish between two different concepts: "a thread bundle successfully completes its computational task" and "a thread bundle exits its computational task." A thread bundle successfully completing its computational task means that it has successfully executed the currently assigned computational task. At this point, the thread bundle remains alive, and its corresponding computational unit can still be used to execute subsequent computational tasks. This situation can be recorded by updating the arrival count. A thread bundle exiting its computational task means that the thread bundle no longer participates in any subsequent computational tasks due to various reasons (such as exiting normally after completing all task blocks, encountering an exception, receiving a termination instruction, or its corresponding computational unit being defective and unusable). This situation is recorded by updating the expectation count and the liveness count. Similarly, when all thread bundles within a thread block exit their computational tasks, the entire thread block exits, meaning that the computational unit corresponding to that thread block can no longer execute computational tasks. The corresponding bit in the mask information for that computational unit should be updated from an available state to an unavailable state.

[0091] In one aspect of this embodiment, the liveness state refers to whether a thread block or thread bundle is still in the normal execution process. When a thread block or thread bundle is in the normal execution state, it is in the liveness state and continues to participate in the execution and synchronization process of the computation task; when a thread block or thread bundle exits the execution of the computation task, it changes from the liveness state to the exit state and no longer participates in subsequent computation tasks and synchronization processes.

[0092] In some embodiments, a thread bundle may exit the execution of a computational task for various reasons. For example, a thread bundle may exit normally after completing all assigned task blocks; or it may exit prematurely due to an exception or by receiving a termination instruction. When a single thread bundle exits execution, it no longer participates in subsequent synchronization, but other thread bundles may continue to execute computational tasks on the computational unit it occupied, so the survival count remains unchanged temporarily. When all thread bundles within a thread block exit the execution of computational tasks, the entire thread block exits, the survival count is decremented by one, and the computational unit corresponding to that thread block is no longer used to execute subsequent computational tasks.

[0093] In some embodiments, the liveness count is set at the same time as the expected count and arrival count, which is when the local scheduler allocates entries for the first thread block group. The local scheduler sets the liveness count to the number of thread blocks contained in the first thread block group. For example, assuming a thread block group contains 4 thread blocks, the liveness count is set to 4. When all thread bundles within any thread block exit execution, the liveness count is decremented by one; when the liveness count reaches zero, it indicates that all thread blocks in the first thread block group have exited, meaning that the computation units corresponding to all thread blocks are no longer available for executing computational tasks.

[0094] The introduction of a liveness count provides an independent and clear criterion for determining when to reclaim entries. When deciding whether to reclaim an entry, both the liveness count and the expected count must be checked simultaneously. Specifically, the liveness count reflects whether there are still thread blocks in the first thread block group capable of executing computational tasks, while the expected count reflects whether there are still thread bundles in the thread block group capable of executing computational tasks. Only when both the liveness count and the expected count are zero does it indicate that there are neither executable thread blocks nor executable thread bundles in the thread block group, at which point the entry can be safely reclaimed.

[0095] It is worth noting that the specific implementation and decrement conditions of the above-mentioned survival count are merely illustrative examples, and the embodiments of this application do not limit them.

[0096] By introducing a liveness count to manage the liveness status of thread blocks separately, the local scheduler can distinguish between the synchronization state at the thread bundle level and the exit state at the thread block level, thereby clarifying the number of currently available computing units in the computing unit group and providing an accurate basis for determining when to reclaim entries.

[0097] In some embodiments, the synchronization request includes a discard flag indicating whether the thread bundle sending the synchronization request has exited execution. Receiving a synchronization request and updating the count information includes: for each thread bundle in the first thread block group, decrementing the count by one upon receiving a synchronization request from the thread bundle; and decrementing the expected count by one upon receiving a synchronization request from the thread bundle and the discard flag indicating that the thread bundle has exited execution.

[0098] In one aspect of this embodiment, the discard flag is an indicator field in the synchronization request, used to distinguish whether the synchronization request is sent after the thread bundle has successfully completed its computation task or when the thread bundle exits execution. For example, when the discard flag is a first value (e.g., 0), it indicates that the thread bundle has successfully completed its computation task and reports its arrival to the local scheduler; when the discard flag is a second value (e.g., 1), it indicates that the thread bundle has exited execution and will no longer participate in the subsequent synchronization process. This embodiment does not limit the specific value and encoding method of the discard flag.

[0099] In one aspect of this embodiment, upon receiving a synchronization request, the local scheduler decrements the arrival count by one, regardless of the state indicated by the discard flag, to indicate that the thread bundle has completed its synchronization report for the current synchronization cycle. When the discard flag indicates that the thread bundle has exited execution, the local scheduler decrements the expectation count by one to dynamically adjust the total number of thread bundles that need to complete synchronization within the current synchronization cycle. Since exiting thread bundles no longer participate in subsequent synchronization, if only the arrival count is decremented without adjusting the expectation count, the expectation count will remain unchanged from its initial value, potentially causing the arrival count to fail to reach the initial value of the expectation count, thus resulting in a synchronization deadlock. By adjusting the expectation count synchronously, the synchronization completion condition can be kept consistent with the actual number of surviving thread bundles.

[0100] In one aspect of this embodiment, the dynamic adjustment of the expectation count ensures that the number of available computing units implied in the query request remains consistent with the actual available resources. When a thread bundle exits execution, the actual available computing resources in the computing unit group decrease accordingly. The adjusted expectation count accurately reflects the number of currently surviving thread bundles, thereby ensuring the accuracy of the quantity information corresponding to subsequent query requests, enabling the global scheduler to issue thread block groups that match the actual available resources of the computing unit group.

[0101] It is worth noting that the specific implementation of the discard flag and the adjustment method of the expected count described above are merely illustrative examples, and the embodiments of this application do not limit these aspects.

[0102] By synchronously reducing the expectation count when a thread bundle exits, the synchronization expectation value can be dynamically adjusted. When a thread bundle exits, the adjusted expectation count accurately reflects the number of thread bundles that have not yet exited execution. This ensures that the quantity information implicit in the query request is consistent with the actual available computing resources, avoiding the global scheduler from issuing mismatched thread block groups due to inaccurate quantity information.

[0103] In some embodiments, the synchronization request is sent by the thread bundle through the computing unit, which maintains a flag bit to prevent the same thread bundle from repeatedly sending the synchronization request within the same synchronization cycle.

[0104] In some embodiments, when the flag corresponding to the thread bundle is in a first state, the computing unit prohibits sending synchronization requests for the thread bundle; when the flag corresponding to the thread bundle is in a second state, the computing unit allows sending synchronization requests for the thread bundle and sets the flag corresponding to the thread bundle to the first state. The first state and the second state are different.

[0105] In one aspect of this embodiment, the flag bit is a status indicator bit maintained internally by the computing unit, used to record whether the corresponding thread bundle has sent a synchronization request within the current synchronization cycle. For example, when the flag bit is in the first state (such as the set state), it indicates that the thread bundle has sent a synchronization request within the current synchronization cycle; when the flag bit is in the second state (such as the cleared state), it indicates that the thread bundle has not sent a synchronization request within the current synchronization cycle. The computing unit allows the thread bundle to send a synchronization request when it completes its computing task or exits execution, and sets the flag bit to the first state after sending. This application embodiment does not limit the specific state value and representation method of the flag bit.

[0106] In one aspect of this embodiment, the same synchronization period refers to the time interval between the start of the current computation task in a thread block group and the completion of synchronization for that thread block group. Within the same synchronization period, each thread bundle only needs to send a synchronization request to the local scheduler once. If the same thread bundle repeatedly sends synchronization requests within the same synchronization period, the arrival count will be decremented multiple times, causing the arrival count to reach zero prematurely, thus incorrectly determining synchronization completion when some thread bundles have not yet completed execution. By maintaining a flag bit in the computation unit and disabling or allowing the sending of synchronization requests based on the flag bit, the aforementioned repeated sending situation can be effectively prevented.

[0107] In some embodiments, the flag bit is reset after the thread block group synchronization is completed. When the local scheduler determines that the thread block group synchronization is complete and broadcasts a synchronization completion signal, the computing unit receives the synchronization completion signal and resets the corresponding flag bit from the first state to the second state, preparing for the next synchronization cycle. The flag bit reset can be performed automatically by hardware or triggered by software instructions; this embodiment does not limit this.

[0108] It is worth noting that the specific implementation methods, state values ​​and reset mechanisms of the above-mentioned flag bits are only illustrative examples, and the embodiments of this application do not limit them.

[0109] By maintaining flags within the computational units to prevent the same thread bundle from repeatedly sending synchronization requests, and controlling the sending of synchronization requests based on the status of the flags, the judgment of synchronization completion will not be prematurely triggered due to repeated counting. This ensures that all thread bundles have actually completed execution when a query request is initiated, and that the information on the number of available computational units implicit in the query request is accurate and reliable. At the same time, the flag state switching mechanism enables precise control over the sending behavior of synchronization requests within each synchronization cycle, ensuring the accuracy of the count information update, thereby improving the reliability of task scheduling.

[0110] In some embodiments, the method further includes: for each thread block in the first thread block group, decrementing the survival count by one if all thread bundles within the thread block have exited execution.

[0111] In one aspect of this embodiment, the condition for decreasing the survival count is that all thread bundles within a thread block exit execution. When a single thread bundle exits, synchronization adjustments are made at the thread bundle level only through the expected count and the arrival count, and the survival count remains unchanged. Only when all thread bundles within a thread block exit execution does the entire thread block exit, and the survival count is decremented by one. Through this mechanism, the survival count can accurately reflect the number of thread blocks still participating in execution in the first thread block group, rather than the number of thread bundles.

[0112] In one aspect of this embodiment, the local scheduler can indirectly determine whether all thread bundles within a thread block have exited execution by observing changes in the expected count or the arrival count. For example, when the local scheduler processes a synchronization request, if all thread bundles within a thread block send synchronization requests with the flag in the second state, the local scheduler can determine that all thread bundles within that thread block have exited, thereby decrementing the survival count by one. Alternatively, the local scheduler can maintain a thread bundle exit counter for each thread block in the table entries. When the thread bundle exit counter of a thread block reaches the total number of thread bundles contained in that thread block, the local scheduler determines that the entire thread block has exited. This application embodiment does not limit the specific method by which the local scheduler determines the overall exit of a thread block.

[0113] In one aspect of this embodiment, if the survival count is still greater than zero after the survival count is decremented, it indicates that there are still surviving thread blocks in the first thread block group, and the local scheduler continues to manage the synchronization state of the first thread block group; if the survival count is decremented to zero, it indicates that all thread blocks in the first thread block group have exited.

[0114] By decrementing the survival count when all thread bundles within a thread block exit, the local scheduler can accurately track the changes in the number of surviving thread blocks in the first thread block group, providing a basis for subsequently determining the number of available computing units in the computing unit group.

[0115] In some embodiments, obtaining the synchronization completion status of the first thread block group based on the counting information includes: determining that the synchronization completion status of the first thread block group is completed when the count reaches zero.

[0116] In one aspect of this embodiment, the arrival count reaching zero indicates that all surviving thread bundles in the first thread block group have completed their current computational tasks and sent synchronization requests. Since the expected count is dynamically adjusted as thread bundles exit, when the arrival count reaches zero, the actual number of arriving thread bundles is consistent with the number of currently surviving thread bundles, indicating that all thread bundles requiring synchronization have arrived. At this point, the local scheduler determines that the synchronization of the first thread block group is complete.

[0117] In some embodiments, after determining that the first thread block group has completed synchronization, the local scheduler broadcasts a synchronization completion signal to all participating computation units in the computation unit group. Upon receiving the synchronization completion signal, the computation unit resets its maintained flags to prepare for the next synchronization cycle. Subsequently, the representative thread bundle designated in the computation unit group initiates a query request to the global scheduler to obtain a new thread block group and continue executing the next round of computation tasks.

[0118] In some embodiments, the local scheduler may determine the synchronization completion of the first thread block group in other ways. For example, after each update of the count information, the local scheduler may compare the arrived count with the expected count, and determine synchronization completion when the two are equal. This application does not limit the specific method for determining synchronization completion.

[0119] It is worth noting that the above-mentioned methods for determining the completion of synchronization and the processing flow after the completion of synchronization are merely illustrative examples, and the embodiments of this application do not limit them.

[0120] By using the arrival count to zero as the condition for synchronization completion, a clear basis for determining synchronization completion is provided, and query requests can be accurately triggered when all surviving thread bundles have completed their computation tasks, further improving the efficiency of task scheduling.

[0121] In some embodiments, after determining that the synchronization completion status of the first thread block group is complete, the method further includes: resetting the arrival count to the desired count.

[0122] In one aspect of this embodiment, the arrival count is reset to the current expected count value after synchronization is completed, in preparation for a new round of synchronization for the computation task corresponding to the next thread block group. Since the expected count may be less than the initial value due to thread bundle exits, the reset arrival count is based on the current number of surviving thread bundles, rather than the initial total number of thread bundles. For example, initially both the expected count and the arrival count are 32. If two thread bundles exit during the current synchronization cycle, the expected count is adjusted to 30; upon synchronization completion, the arrival count is reset to zero, and the reset arrival count is set to 30, consistent with the current number of surviving thread bundles. After the above reset operation is completed, the local scheduler can use this entry to manage the synchronization state of the next thread block group during the execution of its computation task.

[0123] In one aspect of this embodiment, resetting the arrival count allows the same entry to be reused to serve different thread block groups. After a computational unit group obtains the next thread block group through a query request, that thread block group can continue to use the original entries and corresponding identification information. The local scheduler manages the synchronization state of the next thread block group through the reset arrival count. This entry reuse mechanism avoids the overhead of reallocating entries each time a new thread block group is obtained, ensuring that the computational unit group can connect to the computational task of the next thread block group after completing the computational task of the current thread block group, thus realizing the continuous cyclic initiation of query requests. It should be emphasized that the arrival count reset operation can be performed before or after the synchronization completion signal broadcast. This embodiment does not limit the order of the reset operation and the broadcast operation.

[0124] It is worth noting that the above-mentioned timing and method for resetting the arrival count are merely illustrative examples, and the embodiments of this application do not limit these aspects.

[0125] By resetting the arrival count to the expected count after synchronization is complete, the same entry can be reused for the synchronization cycle corresponding to the computation task of the next thread block group, avoiding the additional overhead caused by entry reallocation.

[0126] In some embodiments, the method further includes: reclaiming entries when both the survival count and the expected count are zero.

[0127] In one aspect of this embodiment, the condition for reclaiming an entry is that both the liveness count and the expected count are zero. A liveness count of zero indicates that all thread blocks in the first thread block group have exited the computation task, and an expected count of zero indicates that all thread bundles in the first thread block group have exited the computation task. When both conditions are met simultaneously, it means that no computation unit in the computation unit group is executing the computation task of the first thread block group. At this point, the local scheduler reclaims the entry, releasing the storage space occupied by the entry.

[0128] In some embodiments, the table entry data is stored in a storage unit within the local scheduler. Each storage unit maintains multiple table entries, each corresponding to a currently executing thread block group. The data stored in the table entries includes, but is not limited to, count information such as liveness count, expected count, and arrival count. The local scheduler can access table entries by addressing them using their index numbers.

[0129] For example, in practical application scenarios, the reclamation of table entries mainly occurs in the following two situations: 1. The computation task completes normally, and the computation unit group completely exits. When a computation unit group initiates a query request, if the global scheduler returns a request failure signal, it indicates that there are no thread block groups waiting to be allocated in the scheduling queue. At this time, each thread bundle in the computation unit group executes the exit procedure. As thread bundles and thread blocks gradually exit, both the liveness count and the expected count decrease. When both the liveness count and the expected count decrease to zero, it indicates that all thread blocks and all thread bundles in the first thread block group have exited, and there are no longer any computing resources in the computation unit group executing the computation tasks of the first thread block group. At this time, the local scheduler reclaims the entry and releases the storage space.

[0130] 2. All available computing units in a computing unit group have exited, making it impossible to continue executing subsequent tasks. In scenarios where thread block groups are continuously issued, when a computing unit group successfully obtains a new thread block group through a query request, the new thread block group can continue to be executed by the surviving thread blocks and thread bundles in the same computing unit group. Since the computing resources occupied by the computing unit group are not released, the thread block group identification information corresponding to the computing unit group remains unchanged, and the local scheduler does not need to allocate a new entry for it. The survival count and expected count stored in the original entry reflect the number of currently surviving thread blocks and thread bundles, and the above count information remains valid in the new synchronization cycle. Therefore, after the local scheduler resets the arrival count to the expected count, the entry can continue to be used to manage the synchronization state of the new thread block group. However, during continuous execution, as some thread bundles and thread blocks gradually exit, the number of available computing units in the computing unit group will gradually decrease. When both the survival count and the expected count decrease to zero, it indicates that all surviving thread blocks and thread bundles in the computing unit group have exited, that is, there are no longer any available computing units in the computing unit group that can continue to execute computing tasks. At this point, even if there are still thread blocks to be allocated in the global scheduling queue, this computation unit group can no longer take on any new computation tasks. Therefore, the local scheduler reclaims the entry, and the computation unit group exits execution.

[0131] It is worth noting that the above-mentioned recycling conditions, storage structure and reuse mechanism are only illustrative examples, and the embodiments of this application do not limit the specific recycling time, storage form and reuse method of the entries.

[0132] Through the above mechanism, table entries are continuously reused as long as the computing unit group can continuously acquire computing tasks; table entries are only reclaimed when the computing unit group is not allocated a new thread block group after sending a query request, or when the computing unit group cannot continue to execute computing tasks due to the lack of available computing units. In summary, by clearly defining the reclamation conditions for table entries, the overhead caused by frequent allocation and / or reclamation of table entries is avoided, and the premature reclamation of table entries that could lead to the interruption of the query request chain is also prevented.

[0133] For example, please refer to Figure 3 , Figure 3 This is a schematic diagram of a local scheduler and computing unit group provided in one possible implementation of this application. The diagram illustrates an implementation of an architecture for a computing unit group 130 and its corresponding local scheduler 120. The computing unit group 130 includes multiple computing units (computing unit 0 to computing unit 7). The local scheduler 120 includes an on-chip network interface 121, a local traverser 122, a storage unit 123, and a query module 124. Specifically: In this embodiment, the on-chip network interface 121 is the interface module for communication between the local scheduler 120 and the global scheduler. The on-chip network interface 121 is used to receive the first thread block group issued by the global scheduler and to send query requests to the global scheduler. Furthermore, the on-chip network interface 121 can be implemented as a master interface, a slave interface, or simultaneously support master-slave mode; this embodiment does not limit this implementation.

[0134] In this embodiment, the local traverser 122 is used to split the first thread block group into thread blocks and distribute the split thread blocks to the computing units in the computing unit group 130 for execution. Specifically, the first thread block group can be represented by coordinate information and / or mask information, etc. The coordinate information is used to indicate the position of the first thread block group in the grid, and the mask information is used to indicate the computing units in the computing unit group 130 that can be used to execute the thread blocks in the first thread block group. According to the size of the first thread block group and the corresponding mask information, the local traverser 122 splits the first thread block group into multiple thread blocks and sequentially distributes each thread block to the available computing units indicated by the mask information, so that each available computing unit can execute the corresponding thread block.

[0135] In this embodiment, storage unit 123 is used to allocate entries for the first thread block group and maintain the synchronization status information of the first thread block group in the entries. Storage unit 123 receives synchronization requests from each computing unit in computing unit group 130 and updates the counting information in the entries according to the synchronization requests. Specifically, the synchronization request can be a thread bundle-level synchronization request, that is, after each thread bundle completes its computing task, it sends a synchronization request to storage unit 123 through its computing unit. After determining that the first thread block group has completed synchronization, storage unit 123 sends a synchronization completion signal to query module 124 to notify query module 124 that a query request can be initiated.

[0136] In some embodiments, the mask information of the first thread block group can be directly stored in the corresponding entry of storage unit 123 during transmission, for use by the global scheduler to retrieve the mask information during subsequent queries. Furthermore, the mask information stored in the entry can be updated using the counting information in the synchronization request. For example, when all thread bundles within a thread block corresponding to a certain computing unit exit execution, since that computing unit will no longer be used to execute computing tasks, the bit corresponding to that computing unit in the mask information can be updated from an available state to an unavailable state. The specific methods for obtaining and updating the mask information will be described in detail in subsequent embodiments and will not be repeated here.

[0137] In this embodiment, the query module 124 is used to initiate a query request to the global scheduler through the on-chip network interface 121. In some embodiments, after determining that synchronization is complete, the local scheduler 120 broadcasts a synchronization completion signal to its corresponding computing unit group 130. After receiving the synchronization completion signal, each computing unit in the computing unit group 130 sends a query request to the query module 124 on behalf of one of the computing units (or the thread bundle running on the computing unit), namely computing unit 0 shown in the figure, on behalf of the entire computing unit group 130. Figure 3 Only the calculation unit 0 and the query module 124 have a transmission path for query requests; the other calculation units do not directly send query requests to the query module 124. It is worth noting that the calculation unit representing the initiating query request can be determined in a pre-specified manner or dynamically through hardware arbitration; this embodiment does not limit this approach.

[0138] It is worth noting that, Figure 3The above-described functional division of modules is merely illustrative and is intended to illustrate the task scheduling method applied to the local scheduler proposed in this application. It does not constitute a limitation on the embodiments of this application. For example, the query module 124 can simultaneously perform the function of the local traverser 122, splitting the first thread block group and allocating the thread blocks to each computing unit. The correspondence between the local scheduler 120 and the computing unit group 130 is not limited to the one-to-one correspondence shown in the figure, as long as the local scheduler 120 and the computing unit group 130 have a pre-set correspondence. The local scheduler 120 can be set inside or outside the computing unit group 130. The embodiments of this application do not limit the functional division, implementation form, or physical location of each module in the local scheduler 120.

[0139] In some embodiments, the query request includes identification information of a computing unit group. Based on the query request, quantity information is obtained, including but not limited to: obtaining mask information corresponding to the computing unit group based on the identification information, where one bit in the mask information corresponds to one computing unit in the computing unit group; and determining the number of available computing units in the computing unit group based on the mask information to obtain quantity information.

[0140] In one aspect of this embodiment, the identification information is used to identify a computing unit group. For example, the identification information can be generated when the local scheduler allocates an entry for the first thread block group and distributed to each computing unit within the computing unit group. When a computing unit group initiates a query request, the query request carries the aforementioned identification information, and the global scheduler can locate the corresponding thread block group and its related information based on this identification information. For example, in a GPU, the identification information can be a thread block group index number, which is an entry index number of a storage unit in the local scheduler.

[0141] In one aspect of this embodiment, the mask information is used to indicate the availability status of each computing unit in the computing unit group. One bit in the mask information corresponds to one computing unit in the computing unit group, and the value of the bit indicates whether the corresponding computing unit is available. For example, the mask information can be an 8-bit binary number, with each bit corresponding to one computing unit in the computing unit group. When a bit is "1", it indicates that the corresponding computing unit is available, meaning that the computing unit can be used to execute the thread block in the first thread block group; when a bit is "0", it indicates that the corresponding computing unit is unavailable, meaning that the computing unit does not participate in the current computing task. For example, when the mask information is "11110000", it indicates that computing units 0 to 3 are available, and computing units 4 to 7 are unavailable, meaning that the number of available computing units is 4. For example, when the mask information is "10101010", it indicates that calculation units 0, 2, 4, and 6 are available, while calculation units 1, 3, 5, and 7 are unavailable, meaning that the number of available calculation units is 4. This application does not limit the specific bit width and encoding method of the mask information in its embodiments.

[0142] In one aspect of this embodiment, the global scheduler can obtain the mask information corresponding to the computation unit group based on the identification information in the query request. Specifically, after parsing the identification information from the query request, the global scheduler obtains the corresponding mask information from the storage area storing thread block group information based on the identification information. This storage area can be a storage unit in the local scheduler or a shared storage area accessible to the global scheduler. For example, the global scheduler can send a read request to the storage unit of the local scheduler through the on-chip network, carrying the identification information in the read request. The local scheduler then reads the mask information from the corresponding entry based on the identification information and returns it to the global scheduler. Alternatively, the mask information can be sent to the global scheduler along with the query request, and the global scheduler can directly parse the mask information from the query request without accessing the storage unit of the local scheduler. This embodiment does not limit the specific method and access path for the global scheduler to obtain the mask information.

[0143] In some embodiments, the mask information can be dynamically updated based on the exit status of each thread bundle in the thread block group to ensure that the mask information is consistent with the actual number of available computing units in the computing unit group. As the computing task is executed, some thread bundles in the thread block group may exit execution. When all thread bundles within a thread block have exited, the computing unit corresponding to that thread block no longer participates in subsequent computing task execution, and the bit corresponding to that computing unit in the mask information should be updated from an available state to an unavailable state. For example, the dynamic updating of the mask information can be achieved in the following ways: 1. When updating the count information in the local scheduler's table, the mask information is updated synchronously. When the local scheduler determines that a thread block is exiting entirely based on the decreasing liveness count, it updates the bit in the mask information of the corresponding computation unit of that thread block from the set state to the reset state. The updated mask information is stored in the table entry of the storage unit, so that the global scheduler obtains the mask information that has been updated in real time.

[0144] 2. The query request directly carries the updated mask information. When a computing unit group initiates a query request, it reads the latest mask information from the local scheduler's storage unit and sends it to the global scheduler along with the query request. The global scheduler directly parses the latest mask information from the query request.

[0145] 3. After obtaining the mask information, the global scheduler can further verify or adjust it by combining it with other information. For example, the global scheduler can combine the load status information of the computing unit group and the task execution history information to correct the mask information, so as to more accurately reflect the current actual available resources of the computing unit group.

[0146] It is worth noting that the above-mentioned dynamic update method of mask information is only an illustrative example, and the embodiments of this application do not limit the specific timing, method, or triggering conditions for updating the mask information.

[0147] By dynamically updating the mask information, the global scheduler obtains mask information that accurately reflects the number of currently available computing units in the computing unit group. During continuous task scheduling, the mask information is dynamically adjusted as thread bundles and thread blocks exit, ensuring that the quantity information corresponding to subsequent query requests remains consistent with the actual available resources of the computing unit group. This allows the size of the thread block group issued by the global scheduler each time to match the actual number of available computing units in the computing unit group, improving the accuracy of task scheduling.

[0148] In one aspect of this embodiment, after the global scheduler obtains the mask information, it determines the number of available computing units in the computing unit group based on the number of set bits in the mask information, i.e., it obtains the quantity information. For example, when the mask information is "11110000", the number of set bits is 4, so the number of available computing units is 4; when the mask information is "10101011", the number of set bits is 5, so the number of available computing units is 5. It is worth noting that the specific forms of the above-mentioned identification information and mask information, and the method of obtaining the mask information are only illustrative examples, and this application embodiment does not limit them.

[0149] By obtaining the mask information through the identification information, one bit in the mask information corresponds to one computing unit in the computing unit group, thereby accurately and efficiently determining the number of available computing units in the computing unit group. This ensures that the number of thread blocks in the first thread block group issued is highly matched with the number of available computing units in the computing unit group, improving the accuracy and efficiency of task scheduling.

[0150] In some embodiments, the number of thread blocks in the first thread block group is less than or equal to the number of available computing units in the computing unit group.

[0151] In one aspect of this embodiment, the matching principle between the computing resources of a computing unit group and the computing tasks allocated to the computing unit group can include the following two cases. The first case is that the number of thread blocks in the first thread block group is equal to the number of available computing units in the computing unit group, meaning the task size and resource size are precisely matched, and each available computing unit executes one thread block within a single synchronization cycle. The second case is that the number of thread blocks in the first thread block group is less than the number of available computing units in the computing unit group, meaning the task size is less than the resource size. In this case, each thread block in the first thread block group is allocated to some available computing units for execution, while the remaining available computing units are idle. However, the computing unit group can still support the first thread block group. Through the above matching principle, when there is no thread block group in the global scheduling queue with a number of thread blocks equal to the number of available computing units, the global scheduler can also select a thread block group with a number of thread blocks less than the number of available computing units for distribution, ensuring that computing unit groups with fewer available computing units are not excluded from task allocation due to the lack of precisely matched computing tasks.

[0152] In some embodiments, the matching principle between the computing resources of a computing unit group and the computing tasks allocated to that computing unit group can also have other implementation forms. For example, the matching principle can be that the difference between the number of thread blocks and the number of available computing units is within a preset threshold range, that is, allowing a certain degree of resource redundancy or task overload. Specifically, when the number of available computing units is 8, thread block groups with 6 to 8 thread blocks can all be used as the first thread block group, thereby expanding the matching range and improving the task scheduling rate. Another example is that the matching principle can be that the number of thread blocks and the number of available computing units meet a preset ratio, such as the number of thread blocks being an integer multiple of the number of available computing units, to adapt to different parallel granularity requirements. Yet another example is that the matching principle can be that the number of thread blocks does not exceed the number of available computing units while not falling below a preset minimum number of thread blocks, to ensure the efficiency of task scheduling.

[0153] In some embodiments, the global scheduler can flexibly select different matching principles based on the actual application scenario and scheduling strategy. For example, in scenarios with abundant computing resources, a stricter matching principle (such as the number of thread blocks equaling the number of available computing units) can be adopted to improve resource utilization; in scenarios with scarce computing resources or high task priority, a more lenient matching principle (such as the number of thread blocks being less than the number of available computing units) can be adopted to ensure that computing tasks can be allocated and executed in a timely manner. It is worth noting that the above matching principles are only illustrative examples, and the specific matching relationships in the embodiments of this application are not limited.

[0154] By limiting the matching principle to the number of thread blocks in the first thread block group to less than or equal to the number of available computing units in the computing unit group, even if there is no precise match, the computing unit group with fewer available computing resources can still be assigned an executable thread block group, thereby continuously participating in the computing task and improving the utilization rate of the system's computing resources.

[0155] In some embodiments, the global scheduler includes a global scheduling queue for storing at least one thread block group. The first thread block group is determined based on quantity information, and may include, but is not limited to: If a first candidate thread block group exists in the global scheduling queue, the first candidate thread block group is used as the first thread block group, and the number of thread blocks in the first candidate thread block group is equal to the number of available computing units in the computing unit group; if a first candidate thread block group does not exist in the global scheduling queue but a second candidate thread block group exists, the second candidate thread block group is used as the first thread block group, and the number of thread blocks in the second candidate thread block group is less than the number of available computing units in the computing unit group.

[0156] In one aspect of this embodiment, the global scheduling queue is a storage area in the global scheduler used to store thread block groups to be scheduled. The thread block groups stored in the global scheduling queue can be generated by the traversal unit after traversing the grid. For example, the traversal unit can traverse the grid according to a preset granularity, generate thread block groups of a preset size, and store them in the global scheduling queue so that the global scheduler can determine the first thread block group from it when responding to a query request.

[0157] In one aspect of this embodiment, the candidate thread block group refers to the thread block group stored in the global scheduling queue. The first candidate thread block group is the thread block group whose number of thread blocks is equal to the number of available computing units, and the second candidate thread block group is the thread block group whose number of thread blocks is less than the number of available computing units. When determining the first thread block group, the global scheduler selects the first candidate thread block group first, followed by the second candidate thread block group.

[0158] In some embodiments, the global scheduler can determine whether a first candidate thread block group or a second candidate thread block group is stored in the global scheduling queue by traversing or querying the global scheduling queue. For example, the global scheduler can search for a thread block group in the global scheduling queue whose number of thread blocks is equal to the number of available computing units based on quantity information. If found, it is directly used as the first thread block group. If not found, the global scheduler continues to search for a thread block group in the global scheduling queue whose number of thread blocks is less than the number of available computing units, and uses the found thread block group as the first thread block group. Exemplarily, the global scheduler can select candidate thread block groups from the global scheduling queue in a First-In-First-Out (FIFO) order, a maximum matching priority order, or other preset order. On the other hand, in some embodiments, after selecting a thread block group from the global scheduling queue, the global scheduler can remove the selected thread block group from the global scheduling queue, and the queue head pointer or corresponding queue management information of the global scheduling queue is updated accordingly.

[0159] It is worth noting that the definition, selection order, and selection method of the above-mentioned candidate thread block groups are merely illustrative examples, and the embodiments of this application do not limit them.

[0160] By using the above method, priority is given to allocating thread block groups with the same number of thread blocks as the number of available computing units to computing unit groups, so that computing tasks are fully matched with available computing resources; secondly, thread block groups with fewer thread blocks than the number of available computing units are allocated to computing unit groups, so that computing unit groups with fewer available computing resources can still obtain computing tasks that can be carried, thereby improving the matching efficiency between computing resources and computing tasks while ensuring the matching effect.

[0161] In some embodiments, the method further includes: if there is no first candidate thread block group and a second candidate thread block group in the global scheduling queue, but a third candidate thread block group exists, splitting the third candidate thread block group to obtain at least one first candidate thread block group and / or at least one second candidate thread block group. The number of thread blocks in the third candidate thread block group is greater than the number of available computing units in the computing unit group; the first candidate thread block group or the second candidate thread block group obtained after splitting is determined as the first thread block group.

[0162] In one aspect of this embodiment, the third candidate thread block group refers to a thread block group whose number of thread blocks is greater than the number of available computing units. When the global scheduling queue does not store the first candidate thread block group and the second candidate thread block group, the global scheduler can split the third candidate thread block group to obtain a thread block group whose number of thread blocks is equal to or less than the number of available computing units, thereby satisfying the resource carrying capacity of the computing unit group.

[0163] In some embodiments, exemplarily, please refer to Figure 4 , Figure 4 This is a schematic diagram of a global scheduler provided in one possible implementation of this application. The global scheduler 110 includes a traversal unit 400, a query module 450, a global scheduling queue 460, and an on-chip network interface 470. The traversal unit 400 includes a first traverser 410, a second traverser 420, a first compressor 430, and a second compressor 440. The query module 450 receives query requests from the computing unit group and determines a first thread block group from the global scheduling queue 460 based on the query requests. The on-chip network interface 470 communicates with the local scheduler and the computing unit group; the global scheduler 110 receives query requests and / or issues first thread block groups through the on-chip network interface 470.

[0164] In this embodiment, the first iterator 410 is used to traverse the mesh according to a first granularity and generate a first-granularity thread block group. The first granularity corresponds to a larger number of thread blocks; for example, the first-granularity thread block group may contain 4 thread blocks (such as a 2×2 thread block group). The second iterator 420 is used to split the first-granularity thread block group generated by the first iterator 410 to obtain a second-granularity thread block group. The second granularity corresponds to a smaller number of thread blocks; for example, the second-granularity thread block group may contain 2 thread blocks (such as a 1×2 thread block group), and the second granularity is smaller than the first granularity.

[0165] In some embodiments, the input data for the second iterator 420 is the first granularity thread block group generated and output by the first iterator 410, rather than directly traversing the grid. The second iterator 420 does not continuously perform splitting operations; instead, it triggers the splitting operation only when a second granularity thread block group is needed. Since a first granularity thread block group can be split into multiple second granularity thread block groups, the second iterator 420 does not need to repeat its work if there are still unallocated second granularity thread block groups in the global scheduling queue 460.

[0166] In this embodiment, the first compressor 430 corresponds to the first traverser 410, and the second compressor 440 corresponds to the second traverser 420. The first compressor 430 and the second compressor 440 are respectively used to compress the thread block groups output by the first traverser 410 and the second traverser 420 to save the transmission bandwidth of the on-chip network. The compression process refers to converting the complete three-dimensional coordinate data corresponding to the thread block group into identification information representing the thread block group for transmission. The identification information may include, for example, the coordinates of the first thread block in the thread block group, the number of thread blocks, and mask information, etc., which are not limited in this embodiment. The first granularity thread block group generated by the first traverser 410 and the second granularity thread block group obtained by the second traverser 420 can be compressed by the first compressor 430 and the second compressor 440 respectively and stored in the global scheduling queue 460.

[0167] In this embodiment, the query module 450 is used to receive query requests from computing unit groups. In some embodiments, the query request may carry mask information corresponding to the computing unit group. The query module 450 determines the number of available computing units in the current computing unit group based on the number of set bits in the mask information, thereby determining whether to select a first-granularity thread block group or a second-granularity thread block group from the global scheduling queue 460. For example, during the normal polling and distribution of thread block groups, after the first traverser 410 and / or the second traverser 420 generate the thread block group, it is compressed by the first compressor 430 and / or the second compressor 440 and then directly distributed to the computing unit group via the on-chip network, without needing to go through the global scheduling queue 460 for matching. It should be noted that... Figure 4 The transmission path during normal polling is not shown in the diagram to maintain its simplicity. When the task scheduling process begins with a query request initiated by a computing unit group, the query module 450 prioritizes processing the query request and matches it against the global scheduling queue 460 based on the mask information to determine the first thread block group whose number of thread blocks matches the number of available computing units. Since the query module 450 directly obtains the generated thread block group from the global scheduling queue 460, the relevant information of the thread block group (such as the coordinates of the first thread block, the number of thread blocks, and the corresponding mask information) already exists in the global scheduling queue 460. Therefore, it does not need to undergo compression processing by the first compressor 430 or the second compressor 440 and can be directly distributed to the corresponding computing unit group through the on-chip network. After receiving the first thread block group, the corresponding local traverser of the computing unit group can split the first thread block group into multiple thread blocks and allocate the thread blocks to the corresponding available computing units according to the mask information.

[0168] In some embodiments, the above process may include, but is not limited to: after the query module 450 determines the size of the thread block group to be selected based on the mask information, it searches in the global scheduling queue 460. Assume that the number of available computing units in the current computing unit group is 2, meaning that a first thread block group with 2 thread blocks needs to be selected (e.g., a 1×2 thread block group). The query module 450 first searches for a first candidate thread block group with 2 thread blocks in the global scheduling queue 460, but finds none; it continues to search for a second candidate thread block group with fewer than 2 thread blocks, but also finds none; then it finds a third candidate thread block group with more than 2 thread blocks (e.g., a 2×2 thread block group with 4 thread blocks). At this point, neither the first nor the second candidate thread block group exists in the global scheduling queue 460, but the third candidate thread block group does exist.

[0169] Based on the above, a splitting process is triggered. Specifically, the splitting process may include, but is not limited to, splitting the 2×2 thread block group by the second traverser 420. Since this thread block group was generated by the first traverser 410 at a first granularity (assuming the first granularity is 2×2), the second traverser 420 splits it into multiple thread block groups at a second granularity (assuming the second granularity is 1×2). Each split thread block group contains 2 thread blocks, which is equal to the number of available computing units indicated by the mask information. The query module 450 takes one of the split second-granularity thread block groups as the first thread block group and returns it to the computing unit group through the on-chip network interface 470. The remaining thread block groups after splitting can be stored in the global scheduling queue 460 for subsequent task scheduling. For the subsequent task scheduling process, the thread block groups stored in the global scheduling queue 460 may be the first candidate thread block group, the second candidate thread block group, or the third candidate thread block group, depending on the quantity information obtained according to the query request.

[0170] It is important to emphasize that the first, second, and third candidate thread block groups are categorized based on the comparison between the number of thread blocks in each group and the number of available computing units. For example, when the number of available computing units is 4, a thread block group with 4 thread blocks belongs to the first candidate thread block group; when the number of available computing units becomes 2, the thread block group with 4 thread blocks belongs to the third candidate thread block group. The first and second granularities are two preset thread block group sizes for the traversal unit, and are objective, fixed values. For example, the first granularity is fixed at 2×2 (4 thread blocks), and the second granularity is fixed at 1×2 (2 thread blocks), and does not change with the number of available computing units. These two different categorization criteria are not contradictory. For example, a first candidate thread block group could be either a first-granularity group or a second-granularity group, depending on which granularity matches the current number of available computing units.

[0171] It is worth noting that, Figure 4 This application provides only one exemplary implementation of a global scheduler and does not constitute a limitation on the embodiments thereof. For example, the second traverser 420 in the traversal unit 400 can also directly traverse the grid to generate second-granularity thread block groups; the functions of the first compressor 430 and the second compressor 440 can also be implemented by other modules. This application does not limit the internal structure of the global scheduler, the module division in the traversal unit, the connection method of each module, or the generation and splitting method of thread block groups.

[0172] By splitting thread block groups with a number of thread blocks greater than the number of available computing units, thread block groups with a number of thread blocks equal to or less than the number of available computing units are obtained. This allows the global scheduler to quickly generate a suitable first thread block group even when there is no matching thread block group, thereby avoiding the decrease in computing efficiency caused by computing unit groups being idle due to the lack of matching thread block groups.

[0173] In some embodiments, the method further includes: returning a request failure signal to the computation unit group if the first candidate thread block group, the second candidate thread block group, and the third candidate thread block group do not exist in the global scheduling queue.

[0174] In one aspect of this embodiment, after searching the global scheduling queue, if the global scheduler finds no candidate thread block groups (i.e., no thread block groups exist whose number of thread blocks is equal to, less than, or greater than the number of available computing units), it determines that a new thread block group cannot be allocated to the computing unit group that initiated the query request. In this case, the global scheduler generates a request failure signal and returns it to the computing unit group via the on-chip network. In some embodiments, after receiving the request failure signal, the computing unit group executes an exit procedure. Each thread bundle in the computing unit group gradually exits execution, and as the thread bundle and thread block exit execution, the liveness count and expectation count in the local scheduler gradually decrease until both reach zero. The local scheduler then reclaims the corresponding entries and releases the storage space. It is worth noting that the above-described method of generating and returning the request failure signal is merely an illustrative example, and this embodiment does not limit the scope of the invention.

[0175] By returning a failure signal when none of the candidate thread block groups exist, the computation unit group can exit the computation task in a timely manner, avoiding the waste of computational resources that may be caused by continuous waiting.

[0176] For example, please refer to Figure 5 , Figure 5 This is a schematic diagram of a task scheduling process provided in one possible implementation of this application. For example... Figure 5As shown, the global scheduler 110 and the computing unit group 130 are connected via an on-chip network 140, and the computing unit group 130 corresponds to the local scheduler 120. For a description of the global scheduler 110, please refer to... Figure 4 For a description of the local scheduler 120 and the computing unit group 130, please refer to [link / reference]. Figure 3 . Figure 5 This document describes the complete flow of the task scheduling method provided in the embodiments of this application.

[0177] In this embodiment, after the local scheduler 120 determines that the current thread block group has been synchronized, it notifies its corresponding computing unit group 130 to initiate a query request to the global scheduler 110. The query request is transmitted to the global scheduler 110 via the on-chip network 140. The query request carries identification information, and the global scheduler 110 can obtain the mask information corresponding to the computing unit group 130 based on the identification information, and determine the number of available computing units in the computing unit group 130 based on the mask information. Based on the number of available computing units, the global scheduler 110 determines the first thread block group whose number of thread blocks matches the number of available computing units from the global scheduling queue 460. If there is no first candidate thread block group and no second candidate thread block group in the global scheduling queue 460, but there is a third candidate thread block group, the global scheduler 110 splits the third candidate thread block group through the second traverser 420 in the traversal unit 400 to obtain the first candidate thread block group and / or the second candidate thread block group, and uses it as the first thread block group. When there are no available thread block groups in the global scheduling queue 460, the global scheduler 110 returns a request failure signal (this process is not shown in the figure).

[0178] The global scheduler 110 issues the first thread block group to the computing unit group 130 via the on-chip network 140. After receiving the first thread block group, the local scheduler 120 allocates the thread blocks in the first thread block group to available computing units within the computing unit group 130 for execution. During the execution of the computing task, when a thread bundle in a thread block completes its corresponding computing task, it sends a synchronization request to the local scheduler 120. The local scheduler 120 tracks the synchronization status of each thread bundle using entries and counting information maintained in the storage unit 123. When the local scheduler 120 determines that the current thread block group has completed synchronization based on the counting information, it again notifies the computing unit group 130 to initiate the next query request to the global scheduler 110 to obtain the next thread block group for continued execution.

[0179] The above process is repeated until the computing unit group 130 receives a request failure signal and then exits the execution of the computing task.

[0180] It is worth noting that, Figure 5This illustration merely provides one possible implementation of the task scheduling process in the embodiments of this application and does not constitute a limitation on the embodiments of this application. The execution order of the above steps, the interaction method between modules, and the signal transmission path can all be adjusted according to actual needs. As long as the task scheduling method provided in the embodiments of this application can be implemented, it falls within the protection scope of this application.

[0181] It should also be noted that, in the above embodiments, for ease of understanding, the technical solution provided by this application is described from the perspective of the interaction between the global scheduler and the local scheduler. The steps executed by the global scheduler can be implemented separately as a task scheduling method on the global scheduler side, and the steps executed by the local scheduler can be implemented separately as a task scheduling method on the local scheduler side.

[0182] The following are system embodiments of this application, which can be used to execute the method embodiments of this application. For details not disclosed in the system embodiments of this application, please refer to the method embodiments of this application.

[0183] This application provides a task scheduling system, which includes a global scheduler and at least one local scheduler. The local scheduler corresponds to a computing unit group, and the computing unit group includes at least one computing unit.

[0184] The local scheduler is used to notify the computing unit group to send a query request to the global scheduler.

[0185] The global scheduler is used to obtain quantity information based on the query request. The quantity information indicates the number of available computing units in the computing unit group. Based on the quantity information, the first thread block group is determined, and the number of thread blocks in the first thread block group matches the quantity information. The first thread block group is then issued to the computing unit group.

[0186] The local scheduler is also used to allocate thread blocks in the first thread block group to available computing units in the computing unit group for execution.

[0187] In some embodiments, the query request includes identification information of the computing unit group. The global scheduler is further configured to: obtain mask information corresponding to the computing unit group based on the identification information, wherein one bit in the mask information corresponds to one computing unit in the computing unit group; and determine the number of available computing units in the computing unit group based on the mask information to obtain quantity information.

[0188] In some embodiments, the number of thread blocks in the first thread block group is less than or equal to the number of available computing units in the computing unit group.

[0189] In some embodiments, the global scheduler includes a global scheduling queue for storing at least one thread block group. The global scheduler is further configured to: if a first candidate thread block group exists in the global scheduling queue, determine the first candidate thread block group as the first thread block group, wherein the number of thread blocks in the first candidate thread block group is equal to the number of available computing units in the computing unit group; if a first candidate thread block group does not exist in the global scheduling queue but a second candidate thread block group exists, determine the second candidate thread block group as the first thread block group, wherein the number of thread blocks in the second candidate thread block group is less than the number of available computing units in the computing unit group.

[0190] In some embodiments, the global scheduler further includes a splitting unit. The splitting unit is configured to: split the third candidate thread block group to obtain at least one first candidate thread block group and / or at least one second candidate thread block group when no first candidate thread block group and a second candidate thread block group exist in the global scheduling queue, and / or when a third candidate thread block group exists. The number of thread blocks in the third candidate thread block group is greater than the number of available computing units in the computing unit group. The global scheduler is further configured to: determine the first candidate thread block group or the second candidate thread block group obtained after splitting as the first thread block group.

[0191] In some embodiments, the global scheduler further includes a failure feedback unit. The failure feedback unit is configured to: return a request failure signal to the computation unit group if the first candidate thread block group, the second candidate thread block group, and the third candidate thread block group do not exist in the global scheduling queue.

[0192] In some embodiments, the local scheduler further includes a synchronization determination unit. The synchronization determination unit is configured to: receive a synchronization request from at least one thread bundle in a first thread block group, the synchronization request being sent by the computation unit of the assigned thread bundle after completing its corresponding computation task; obtain the synchronization completion status of the first thread block group based on the synchronization request; and, if the synchronization completion status is "synchronization completed," notify the computation unit group to initiate the next query request to the global scheduler.

[0193] In some embodiments, the local scheduler further includes a setting unit. The setting unit is configured to: allocate entries for the first thread block group and set counting information in the entries, the counting information being used to record the synchronization status of the first thread block group. The synchronization determination unit is further configured to: update the counting information according to a synchronization request; and obtain the synchronization completion status of the first thread block group based on the counting information.

[0194] In some embodiments, the counting information includes an expected count and an arrival count. The expected count indicates the number of thread bundles in the first thread block group that need to complete synchronization, and the arrival count indicates the number of thread bundles in the first thread block group that have already completed synchronization. The setting unit is further configured to: set the expected count and the arrival count to the total number of thread bundles contained in the first thread block group.

[0195] In some embodiments, the synchronization request includes a discard flag, which indicates whether the thread bundle that sent the synchronization request has exited execution. The synchronization determination unit is further configured to: for each thread bundle in the first thread block group, decrement the arrival count by one when a synchronization request is received from the thread bundle; and decrement the expectation count by one when a synchronization request is received from the thread bundle and the discard flag indicates that the thread bundle has exited execution.

[0196] In some embodiments, the synchronization determination unit is further configured to: determine that the synchronization completion status of the first thread block group is completed when the count reaches zero.

[0197] In some embodiments, the local scheduler further includes a reset unit. The reset unit is configured to: after determining that the synchronization completion status of the first thread block group is complete, reset the arrival count to the desired count.

[0198] In some embodiments, the counting information further includes a survival count, which indicates the number of thread blocks in the first thread block group that have not exited execution. The setting unit is further configured to: set the survival count to the total number of thread blocks in the first thread block group.

[0199] In some embodiments, the local scheduler further includes a reclamation unit. The reclamation unit is configured to: reclamate entries when both the liveness count and the expected count are zero.

[0200] In some embodiments, the synchronization request is sent by the thread bundle through the computing unit, which maintains a flag bit to prevent the same thread bundle from repeatedly sending the synchronization request within the same synchronization cycle.

[0201] In some embodiments, when the flag corresponding to the thread bundle is in a first state, the computing unit prohibits sending synchronization requests for the thread bundle; when the flag corresponding to the thread bundle is in a second state, the computing unit allows sending synchronization requests for the thread bundle and sets the flag corresponding to the thread bundle to the first state, wherein the first state and the second state are different.

[0202] In the task scheduling system provided in this application embodiment, the local scheduler notifies the computing unit group to initiate a query request to the global scheduler. Based on the query request, the global scheduler obtains information on the number of available computing units in the computing unit group, determines a first thread block group based on this information, and distributes the first thread block group to the computing unit group. The local scheduler then allocates the thread blocks in the first thread block group to the available computing units in the computing unit group for execution. Therefore, even when the number of available computing units in a computing unit group is small, the global scheduler can still distribute a suitable thread block group, enabling the computing unit group to execute its computing tasks normally. In summary, the task scheduling system provided in this application embodiment enables computing unit groups with a small number of available computing units to execute computing tasks normally, thereby improving the utilization rate of the system's computing resources.

[0203] It should be noted that the system provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0204] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0205] For example, please refer to Figure 6 , Figure 6 This is a structural block diagram of a global scheduler provided in one possible implementation of this application. The global scheduler 110 includes a receiving unit 610, a processing unit 620, and a sending unit 630.

[0206] The receiving unit 610 is configured to receive a query request from a group of computing units, the group of computing units including at least one computing unit. The processing unit 620 is configured to obtain quantity information according to a query request, the quantity information being used to indicate the number of available computing units in the computing unit group; and to determine a first thread block group based on the quantity information, wherein the number of thread blocks in the first thread block group matches the quantity information. Sending unit 630 is configured to send the first thread block group to the computing unit group.

[0207] In some embodiments, the query request includes identification information of a computing unit group. The processing unit 620 is further configured to: obtain mask information corresponding to the computing unit group based on the identification information, where one bit in the mask information corresponds to one computing unit in the computing unit group; and determine the number of available computing units in the computing unit group based on the mask information to obtain quantity information.

[0208] In some embodiments, the number of thread blocks in the first thread block group is less than or equal to the number of available computing units in the computing unit group.

[0209] In some embodiments, the global scheduler 110 further includes a global scheduling queue (not shown), which stores at least one thread block group. The processing unit 620 is further configured to: if a first candidate thread block group exists in the global scheduling queue, designate the first candidate thread block group as the first thread block group, wherein the number of thread blocks in the first candidate thread block group is equal to the number of available computing units in the computing unit group; if a first candidate thread block group does not exist in the global scheduling queue but a second candidate thread block group exists, designate the second candidate thread block group as the first thread block group, wherein the number of thread blocks in the second candidate thread block group is less than the number of available computing units in the computing unit group.

[0210] In some embodiments, the global scheduler 110 further includes a splitting unit 640. The splitting unit 640 is configured to split the third candidate thread block group to obtain at least one first candidate thread block group and / or at least one second candidate thread block group when a first candidate thread block group and a second candidate thread block group do not exist in the global scheduling queue, but a third candidate thread block group does exist. The number of thread blocks in the third candidate thread block group is greater than the number of available computing units in the computing unit group.

[0211] The processing unit 620 is also configured to determine the first candidate thread block group or the second candidate thread block group obtained after splitting as the first thread block group.

[0212] In some embodiments, the global scheduler 110 further includes a signal return unit (not shown). The signal return unit is configured to return a request failure signal to the computation unit group if the first candidate thread block group, the second candidate thread block group, and the third candidate thread block group are not present in the global scheduling queue.

[0213] For example, please refer to Figure 7 , Figure 7 This is a structural block diagram of a local scheduler provided in one possible implementation of this application. The local scheduler 120 includes a sending unit 710 and a processing unit 720.

[0214] Sending unit 710 is configured to notify the computing unit group to initiate a query request to the global scheduler to instruct the global scheduler to issue a first thread block group to the computing unit group, wherein the number of thread blocks in the first thread block group matches the number of available computing units in the computing unit group.

[0215] Processing unit 720 is configured to allocate thread blocks in the first thread block group to available computing units in the computing unit group for execution.

[0216] In some embodiments, the local scheduler 120 further includes a synchronization determination unit 730. The synchronization determination unit 730 is configured to: receive a synchronization request from at least one thread bundle in a first thread block group, the synchronization request being sent by the computing unit of the assigned thread bundle after completing the corresponding computing task; obtain the synchronization completion status of the first thread block group according to the synchronization request; and, if the synchronization completion status is that synchronization has been completed, notify the computing unit group to initiate the next query request to the global scheduler.

[0217] In some embodiments, the local scheduler 120 further includes a setting unit (not shown). The setting unit is configured to allocate entries for the first thread block group and set counter information in the entries, the counter information being used to record the synchronization state of the first thread block group.

[0218] The synchronization determination unit 730 is also configured to: update the counting information according to the synchronization request; and obtain the synchronization completion status of the first thread block group according to the counting information.

[0219] In some embodiments, the counting information includes an expected count and an arrival count. The expected count indicates the number of thread bundles in the first thread block group that need to complete synchronization, and the arrival count indicates the number of thread bundles in the first thread block group that have already completed synchronization. The setting unit is further configured to set the expected count and the arrival count to the total number of thread bundles contained in the first thread block group.

[0220] In some embodiments, the synchronization request includes a discard flag indicating whether the thread bundle that sent the synchronization request has exited execution. The synchronization determination unit 730 is further configured to: for each thread bundle in the first thread block group, decrement the arrival count by one upon receiving a synchronization request from the thread bundle; and decrement the expectation count by one upon receiving a synchronization request from the thread bundle and the discard flag indicating that the thread bundle has exited execution.

[0221] In some embodiments, the synchronization determination unit 730 is further configured to determine that the synchronization completion status of the first thread block group is completed when the count reaches zero.

[0222] In some embodiments, the local scheduler 120 further includes a count reset unit (not shown). The count reset unit is configured to reset the arrival count to the desired count.

[0223] In some embodiments, the setting unit is further configured to set the liveness count to the total number of thread blocks in the first thread block group.

[0224] In some embodiments, the synchronization determination unit 730 is further configured to decrement the survival count by one for each thread block in the first thread block group if all thread bundles within the thread block have exited execution.

[0225] In some embodiments, the local scheduler 120 further includes a reclamation unit (not shown). The reclamation unit is configured to reclamation of entries when both the live count and the expected count are zero.

[0226] In some embodiments, the synchronization request is sent by the thread bundle through the computing unit, which maintains a flag bit to prevent the same thread bundle from repeatedly sending the synchronization request within the same synchronization cycle.

[0227] In some embodiments, when the flag corresponding to the thread bundle is in a first state, the computing unit prohibits sending synchronization requests for the thread bundle; when the flag corresponding to the thread bundle is in a second state, the computing unit allows sending synchronization requests for the thread bundle and sets the flag corresponding to the thread bundle to the first state, wherein the first state and the second state are different.

[0228] The local scheduler provided in this application embodiment is configured to notify the computing unit group to initiate a query request to the global scheduler. The global scheduler provided in this application embodiment is configured to: obtain information indicating the number of available computing units in the computing unit group based on the query request; determine a first thread block group based on the number information; and distribute the first thread block group to the computing unit group. The local scheduler is also configured to allocate the thread blocks in the first thread block group to the available computing units in the computing unit group for execution. Therefore, even when the number of available computing units in a computing unit group is small, the global scheduler can still distribute an appropriate thread block group to it, enabling the computing unit group to execute computing tasks normally. In summary, the technical solution provided in this application embodiment enables computing unit groups with a small number of available computing units to execute computing tasks normally, thereby improving the utilization rate of system computing resources.

[0229] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0230] For example, please refer to Figure 8 , Figure 8 This is a structural block diagram of a computer device provided in one possible implementation of this application. The computer device 800 can be any electronic device with data computing, processing, and storage functions. The computer device 800 can be used to implement the task scheduling method provided in the above embodiments.

[0231] Typically, computer device 800 may include a processor 810 and a memory 820.

[0232] Processor 810 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 810 may be implemented using at least one hardware form selected from DSP, FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). Processor 810 may also include a main processor and a coprocessor. The main processor, also known as the CPU, is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 810 may integrate a GPU, which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 810 may also include an AI processor, which is used to handle computational operations related to machine learning.

[0233] The memory 820 may include one or more computer-readable storage media, which may be non-transitory. The memory 820 may also include high-speed random access memory and NVM (Non-Virtual Machine). Volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage medium in memory 820 is used to store a computer program configured to be executed by one or more processors to implement the task scheduling method described above.

[0234] Those skilled in the art will understand that Figure 8 The structure shown does not constitute a limitation on the computer device 800, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0235] In an illustrative embodiment, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor of a computer device, implements the task scheduling method described above. Optionally, the computer-readable storage medium may be a ROM (Read-Only Memory), RAM (Random Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk, or optical data storage device, etc.

[0236] In an exemplary embodiment, a chip is also provided, the chip including programmable logic circuitry and / or program instructions, the programmable logic circuitry and / or program instructions being stored in a computer-readable storage medium. A processor of a computer device reads the programmable logic circuitry and / or program instructions from the computer-readable storage medium, and the processor executes the programmable logic circuitry and / or program instructions, causing the computer device to perform the task scheduling method described above.

[0237] In an exemplary embodiment, a computer program product is also provided, which includes a computer program that is loaded and executed by a processor to implement the task scheduling method described above.

[0238] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0239] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A task scheduling method, characterized in that, Applied to a global scheduler, the method includes: Receive a query request from a computing unit group, the computing unit group including at least one computing unit; Based on the query request, quantity information is obtained, which indicates the number of available computing units in the computing unit group; Based on the quantity information, a first thread block group is determined, wherein the number of thread blocks in the first thread block group matches the quantity information; The first thread block group is issued to the computing unit group.

2. The method according to claim 1, characterized in that, The query request includes the identification information of the computing unit group; The step of obtaining quantity information according to the query request includes: Based on the identification information, obtain the mask information corresponding to the computing unit group, where one bit in the mask information corresponds to one computing unit in the computing unit group; Based on the mask information, the number of available computing units in the computing unit group is determined to obtain the quantity information.

3. The method according to claim 1, characterized in that, The number of thread blocks in the first thread block group is less than or equal to the number of available computing units in the computing unit group.

4. The method according to any one of claims 1 to 3, characterized in that, The global scheduler includes a global scheduling queue, which is used to store at least one thread block group; Determining the first thread block group based on the quantity information includes: If a first candidate thread block group exists in the global scheduling queue, the first candidate thread block group is determined as the first thread block group, and the number of thread blocks in the first candidate thread block group is equal to the number of available computing units in the computing unit group. If the first candidate thread block group does not exist in the global scheduling queue but the second candidate thread block group exists, the second candidate thread block group is determined as the first thread block group, and the number of thread blocks in the second candidate thread block group is less than the number of available computing units in the computing unit group.

5. The method according to claim 4, characterized in that, The method further includes: If neither the first candidate thread block group nor the second candidate thread block group exists in the global scheduling queue, but a third candidate thread block group exists, the third candidate thread block group is split to obtain at least one first candidate thread block group and / or at least one second candidate thread block group; wherein, the number of thread blocks in the third candidate thread block group is greater than the number of available computing units in the computing unit group; The first candidate thread block group or the second candidate thread block group obtained after splitting is determined as the first thread block group.

6. The method according to claim 5, characterized in that, The method further includes: If the first candidate thread block group, the second candidate thread block group, and the third candidate thread block group are not present in the global scheduling queue, a request failure signal is returned to the computing unit group.

7. A task scheduling method, characterized in that, Applied to a local scheduler, the local scheduler corresponding to a group of computational units, the group of computational units including at least one computational unit, the method includes: The computing unit group is notified to initiate a query request to the global scheduler. The query request is used to trigger the global scheduler to issue a first thread block group to the computing unit group. The number of thread blocks in the first thread block group matches the number of available computing units in the computing unit group. The thread blocks in the first thread block group are assigned to the available computing units in the computing unit group for execution.

8. The method according to claim 7, characterized in that, The method further includes: Receive synchronization requests from at least one thread bundle in the first thread block group, wherein the synchronization request is sent by the computing unit assigned the thread bundle after completing the corresponding computing task; Based on the synchronization request, obtain the synchronization completion status of the first thread block group; If the synchronization completion status is "synchronization completed", the computing unit group is notified to initiate the next query request to the global scheduler.

9. The method according to claim 8, characterized in that, Before receiving a synchronization request from at least one thread bundle in the first thread block group, the method further includes: Assign a table entry to the first thread block group and set the counting information in the table entry. The counting information is used to record the synchronization status of the first thread block group. The step of obtaining the synchronization completion status of the first thread block group according to the synchronization request includes: Update the counting information according to the synchronization request; Based on the counting information, the synchronization completion status of the first thread block group is obtained.

10. The method according to claim 9, characterized in that, The counting information includes expected count and arrival count; wherein, the expected count is used to indicate the number of thread bundles in the first thread block group that need to complete synchronization; the arrival count is used to indicate the number of thread bundles in the first thread block group that have completed synchronization; Setting the counting information in the table entry includes: Set the expected count and the arrival count to the total number of thread bundles contained in the first thread block group.

11. The method according to claim 10, characterized in that, The synchronization request includes a discard flag, which is used to indicate whether the thread bundle that sent the synchronization request has exited execution; The step of updating the counting information according to the synchronization request includes: For each thread bundle in the first thread block group, upon receiving a synchronization request from the thread bundle, the arrival count is decremented by one; Upon receiving a synchronization request from the thread bundle, and with the discard flag indicating that the thread bundle is exiting execution, the expectation count is decremented by one.

12. The method according to claim 11, characterized in that, The step of obtaining the synchronization completion status of the first thread block group based on the counting information includes: If the count reaches zero, the synchronization completion status of the first thread block group is determined to be completed.

13. The method according to claim 12, characterized in that, After determining that the synchronization completion status of the first thread block group is complete, the method further includes: Reset the arrival count to the expected count.

14. The method according to claim 10, characterized in that, The counting information also includes a survival count, which is used to indicate the number of thread blocks in the first thread block group that have not exited execution; Setting the counting information in the table entry includes: Set the survival count to the total number of thread blocks in the first thread block group.

15. The method according to claim 14, characterized in that, The method further includes: For each thread block in the first thread block group, if all thread bundles within the thread block exit execution, the survival count is decremented by one.

16. The method according to claim 15, characterized in that, The method further includes: If both the survival count and the expected count are zero, the entry is reclaimed.

17. The method according to any one of claims 8 to 16, characterized in that, The synchronization request is sent by the thread bundle through the computing unit. The computing unit maintains a flag bit to prevent the same thread bundle from repeatedly sending the synchronization request within the same synchronization cycle.

18. The method according to claim 17, characterized in that, When the flag corresponding to the thread bundle is in the first state, the computing unit is prohibited from sending the synchronization request for the thread bundle; When the flag corresponding to the thread bundle is in the second state, the computing unit allows the sending of the synchronization request for the thread bundle and sets the flag corresponding to the thread bundle to the first state, which is different from the second state.

19. A task scheduling system, characterized in that, The system includes a global scheduler and at least one local scheduler; wherein, the local scheduler corresponds to a computing unit group, and the computing unit group includes at least one computing unit; The local scheduler is used to notify the computing unit group to initiate a query request to the global scheduler; The global scheduler is configured to obtain quantity information based on the query request, the quantity information indicating the number of available computing units in the computing unit group; determine a first thread block group based on the quantity information, the number of thread blocks in the first thread block group matching the quantity information; and issue the first thread block group to the computing unit group. The local scheduler is also used to allocate thread blocks in the first thread block group to available computing units in the computing unit group for execution.

20. A global scheduler, characterized in that, The global scheduler includes: A receiving unit is configured to receive a query request from a group of computing units, the group of computing units including at least one computing unit. The processing unit is configured to obtain quantity information based on the query request, the quantity information indicating the number of available computing units in the computing unit group; and determine a first thread block group based on the quantity information, wherein the number of thread blocks in the first thread block group matches the quantity information. The sending unit is configured to send the first thread block group to the computing unit group.

21. A local scheduler, characterized in that, The local scheduler corresponds to a computing unit group, the computing unit group including at least one computing unit, and the local scheduler includes: The sending unit is configured to notify the computing unit group to initiate a query request to the global scheduler to instruct the global scheduler to issue a first thread block group to the computing unit group, wherein the number of thread blocks in the first thread block group matches the number of available computing units in the computing unit group; The processing unit is configured to allocate thread blocks from the first thread block group to available computing units in the computing unit group for execution.

22. A chip, characterized in that, The chip includes programmable logic circuitry and / or program instructions, which, when the chip is running, are used to implement the method as described in any one of claims 1 to 6, or the method as described in any one of claims 7 to 18.

23. A computer program product, characterized in that, The computer program product includes a computer program that is loaded and executed by a processor to implement the method as claimed in any one of claims 1 to 6, or the method as claimed in any one of claims 7 to 18.

Images