Task dispatch system and computing chip

Abstract

The application relates to the technical field of chip micro-architecture, and provides a task dispatching system and a computing chip, wherein the system comprises a central controller and a plurality of computing units; the central controller comprises a queue manager, the queue manager is used for managing a task queue in the central controller; the task queue stores the identities of a plurality of to-be-processed task blocks; the computing unit is used for sending a task request to the queue manager when any task block is completed; the queue manager is used for responding to the task request, obtaining the identity of a target task block to be processed from the managed task queue, and returning the identity of the target task block to the computing unit corresponding to the task request; and the computing unit is further used for receiving the identity of the target task block and performing a task based on the identity of the target task block, so that the computing resources of the chip are fully utilized, the "tail effect" that other computing units are forced to idle and wait due to long time consumption of individual tasks is avoided, and the overall operation performance is improved.

Description

Technical Field

[0001] This invention relates to the field of chip microarchitecture technology, and more particularly to a task dispatch system and a computing chip. Background Technology

[0002] In cutting-edge fields such as artificial intelligence, graphics processing, and high-performance computing, fully leveraging the parallel processing capabilities of computing chips to improve overall chip performance has always been a core objective. To achieve this, the key lies in ensuring that all computing units on the chip operate at high efficiency, accelerating the execution efficiency of various computational tasks such as matrix multiplication and convolution by increasing parallelism.

[0003] In existing technologies, a static scheduling strategy is typically employed. Before execution, a computational task is broken down into smaller task blocks and statically allocated to different computing units or groups of computing units. After allocation, each computing unit group executes the task in parallel. However, research has revealed significant shortcomings in this static allocation method. Specifically, due to differences in computational complexity among different task blocks, or uneven memory access latency caused by different data locations in memory, the completion times of each computing unit group vary. Some computing unit groups complete their tasks early but remain idle, resulting in scheduling imbalances. The slowest computing unit group becomes the execution bottleneck, creating a "tailing effect," leading to wasted computing resources and overall performance degradation. Summary of the Invention

[0004] This invention provides a task dispatch system and a computing chip to solve the problem of wasted computing resources caused by static task allocation in existing parallel computing chips.

[0005] The present invention provides a task dispatch system, including a central controller and multiple computing units; the central controller includes a queue manager, which manages the task queue in the central controller, and the task queue stores the identifiers of multiple task blocks to be processed;

[0006] The computing unit is used to send a task request to the queue manager upon completion of any task block;

[0007] The queue manager is used to respond to the task request, obtain the identifier of the target task block to be processed from the managed task queue, and return the identifier of the target task block to the computing unit corresponding to the task request.

[0008] The computing unit is also used to receive the identifier of the target task block and perform task execution based on the identifier of the target task block.

[0009] According to a task dispatching system provided by the present invention, the computing unit is specifically used to send the task request by performing a write operation to a preset register address in the central controller when any task block is completed.

[0010] The write operation is used to trigger the queue manager to assign tasks.

[0011] According to a task dispatching system provided by the present invention, a shared storage area is further included; the shared storage area can be accessed by the queue manager and the computing unit.

[0012] The queue manager is specifically used to write the identifier of the target task block into the storage location corresponding to the computing unit in the shared storage area;

[0013] The computing unit is further configured to read the identifier of the target task block from the storage location in the shared storage area, and store the identifier of the target task block in the identifier location in the local cache; the identifier location is used to store the identifier of the task block to be processed by the corresponding computing unit.

[0014] According to a task dispatch system provided by the present invention, the computing unit is further configured to execute a memory barrier instruction before reading the identifier of the target task block from the storage location in the shared storage area;

[0015] The memory barrier instruction is used to invalidate the cached data at the identified location in the local cache of the computing unit, and to enable the computing unit to directly access the shared storage area when obtaining the identifier of the target task block.

[0016] According to a task dispatching system provided by the present invention, the computing unit is further configured to poll the storage location in the shared storage area, detect whether the queue manager has written the identifier of the target task block into the storage location; if so, read the identifier of the target task block from the storage location and terminate the polling.

[0017] According to a task dispatching system provided by the present invention, the queue manager is further configured to detect whether the managed task queue is empty; if so, it returns a task end identifier to the computing unit that issued the task request.

[0018] The task end flag is used to stop the computing unit from making task requests.

[0019] According to a task dispatching system provided by the present invention, the queue manager is specifically used to manage the task queue through a head pointer and a tail pointer;

[0020] The head pointer indicates the position of the identifier of the next target task block in the task queue; the tail pointer indicates the position of the identifier of the last unprocessed task block in the task queue.

[0021] The queue manager is also used to update the head pointer after obtaining the identifier of the target task block to be processed from the managed task queue.

[0022] According to a task dispatching system provided by the present invention, the computing unit is further configured to determine the data storage address of the task data corresponding to the target task block based on the identifier of the target task block, read the task data from the data storage address, and perform task execution based on the task data.

[0023] According to a task assignment system provided by the present invention, the computing unit is further configured to determine a global task block identifier based on the identifier of the target task block and the offset of the computing unit in its respective computing unit group; and determine the data storage address based on the global task block identifier.

[0024] The present invention also provides a computing chip including the task assignment system as described in any of the preceding claims.

[0025] The task dispatch system and computing chip provided by this invention manage the task queue containing task block identifiers by setting a queue manager in the central controller, and establish a closed-loop workflow in which computing units actively initiate requests after completing tasks, and the queue manager dynamically dispatches task identifiers. This connects all computing units to a common task pool, realizing dynamic load balancing of "the capable do more work", fundamentally avoiding the "tailing effect" of other computing units being forced to wait due to the long time consumption of individual tasks. This allows the parallel computing resources of the chip to be continuously and fully utilized, thereby improving the overall computing performance. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram of the task assignment system provided by the present invention;

[0028] Figure 2 This is an example diagram of the traditional task assignment process provided by the present invention;

[0029] Figure 3This is an interactive example diagram of the task assignment process provided by the present invention;

[0030] Figure 4 This is an example diagram of the computing chip provided by the present invention;

[0031] Figure 5 This is a schematic diagram of the computing chip provided by the present invention.

[0032] Figure label:

[0033] 110: Central Controller; 120: Computing Unit; 111: Queue Manager; 112: Task Queue; 510: Task Dispatch System. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0035] In artificial intelligence, graphics processing, and other high-performance computing fields, one of the core pursuits is to maximize the parallel processing capabilities of computing chips. This typically means ensuring that all computing units (cu) on the chip are operating at high efficiency, thereby accelerating the execution speed of various computing tasks, such as matrix multiplication and convolution, by increasing parallelism, thus optimizing and improving the overall performance of the chip.

[0036] To achieve this goal, existing technologies typically employ a static scheduling strategy. Specifically, before execution, a computational workload is pre-divided into multiple smaller-granularity task blocks (tb). Subsequently, these task blocks are statically assigned to different computational units (cu) or computational unit groups (gcu) on the chip. Figure 2 This is an example diagram of the traditional task assignment process provided by the present invention, such as... Figure 2 As shown, the current task assignment uses a static binding method, meaning task block 0 (tb0) is always assigned to gcu0, task block 1 (tb1) is always assigned to gcu1, and so on. After all task blocks have been assigned, each gcu begins to execute its assigned task in parallel.

[0037] However, research has revealed significant drawbacks to this static binding method. Specifically, in actual operation, differences in the computational complexity of different task blocks, or uneven memory access latency due to the different locations of the data they need to access in memory, can cause varying times for each GCU to complete its task. When some GCUs have already completed their tasks, they cannot assist other busy GCUs and can only enter an idle waiting state. This phenomenon caused by the uneven scheduling among GCUs makes the slowest GCU the bottleneck of the entire workload execution, creating a "tail effect." Ultimately, this leads to a serious waste of computing resources and a significant decline in overall performance, contradicting the original intention of fully utilizing computing resources to improve performance.

[0038] In response, this invention provides a task assignment system that aims to solve the problem of wasted computing resources caused by static task allocation in current parallel computing chips through a dynamic scheduling mechanism at the hardware level, thereby achieving full utilization of chip computing resources, eliminating the "tail effect", and improving overall computing performance.

[0039] Figure 1 This is a schematic diagram of the task assignment system provided by the present invention, as shown below. Figure 1 As shown, this system can be integrated into various high-performance computing chips, such as graphics processing units (GPUs) and artificial intelligence (AI) accelerators. The system includes a central controller 110 as the scheduling hub and multiple computing units 120 as the main computing components. The central controller can be implemented as a dedicated control processor core within the chip, while the computing units are the physical units that execute specific computing tasks. The central controller includes a queue manager 111, which manages the task queues 112 within the central controller. The task queues store identifiers for multiple pending task blocks.

[0040] The computing unit is used to send a task request to the queue manager upon completion of any task block;

[0041] The queue manager is used to respond to task requests, retrieve the identifier of the target task block to be processed from the task queue it manages, and return the identifier of the target task block to the computing unit corresponding to the task request.

[0042] The computing unit is also used to receive the identifier of the target task block and perform task execution based on the identifier of the target task block.

[0043] Specifically, the core of this invention lies in constructing a dynamic scheduling mechanism that actively pulls tasks from a task pool, replacing the traditional, inefficient static allocation mode. To efficiently implement this mechanism at the hardware level, a key functional module, the queue manager (QM), is integrated within the central controller (CP). This queue manager is a dedicated hardware logic unit whose core responsibility is to manage one or more task queues located within the central controller. These task queues constitute the task pool, which can physically be a high-speed on-chip memory, such as static random access memory (SRAM).

[0044] To achieve lightweight and efficient scheduling, this task queue does not store complete task data, but instead stores identifiers for multiple task blocks to be processed. Here, a task block (tb) refers to a smaller-granularity computational task obtained by splitting a large computational load. The task block identifier (tb id, threadblock identity document) is a lightweight symbol that can uniquely identify each task block, such as an integer (id, identity document).

[0045] In detail, in this embodiment of the invention, the entire dynamic scheduling process is a cyclical closed loop, specifically:

[0046] When any computing unit completes its currently executing task block and enters an idle state, it does not stop waiting for other computing units. Instead, it immediately triggers a crucial action: sending a task request to the queue manager in the central controller. This task request is a clear signal designed to inform the scheduling center, "I am idle and request a new task." The method of sending this request can be highly efficient. For example, a computing unit can perform a write operation to a specific register address bound to the queue manager; this action can be captured instantaneously by the queue manager, with a latency far lower than traditional software interrupts.

[0047] Upon receiving a task request from a computing unit, the queue manager responds immediately by accessing its managed task queue to retrieve the identifier of a target task block awaiting processing. Specifically, the queue manager typically retrieves the identifier of an unassigned task block from the head of the queue. Subsequently, the queue manager returns this target task block identifier to the requesting computing unit. This return process also prioritizes efficiency; for example, the queue manager can directly write the identifier to a pre-agreed shared memory address or dedicated register for the computing unit to read.

[0048] After this, the computing unit receives the identifier of the target task block returned by the queue manager through polling or other methods. Once the identifier is successfully obtained, the computing unit will perform subsequent task execution based on the identifier, that is, execute the target task block corresponding to the identifier. This process typically includes: first, the computing unit calculates the data to be processed, that is, the exact location of the task data corresponding to the target task block in global memory, based on the identifier and preset addressing rules; then, the computing unit retrieves the data from that location and starts the computation of the target task block.

[0049] After completing this target task block, the computing unit will repeat the above action of sending task requests, thus seamlessly entering the loop of the next task request, task assignment and task execution, until all task blocks in the task queue have been processed.

[0050] The task dispatch system provided by this invention manages task queues containing task block identifiers by setting up a queue manager in the central controller, and establishes a closed-loop workflow in which computing units actively initiate requests after completing tasks, and the queue manager dynamically dispatches task identifiers. This connects all computing units to a common task pool, achieving dynamic load balancing of "the capable do more work," fundamentally avoiding the "tailing effect" where other computing units are forced to wait due to the long time consumption of individual tasks. This ensures that the parallel computing resources of the chip are continuously and fully utilized, thereby improving the overall computing performance.

[0051] Based on the above embodiments, the computing unit is specifically used to send a task request by performing a write operation to a preset register address in the central controller when any task block is completed.

[0052] Write operations are used to trigger the queue manager to assign tasks.

[0053] Specifically, in order to achieve extremely low latency and extremely low system overhead in the task request process, this embodiment of the invention proposes a triggering mechanism based on direct hardware communication.

[0054] In detail, after completing the currently executing task block, the computing unit is configured to perform a write operation to a preset register address in the central controller, thereby sending a task request. This preset register address is a specific address predefined at the hardware level and associated with the queue manager function. This address is known to all computing units and serves as a dedicated "mailbox" or "button" for event notifications between the computing units and the central controller.

[0055] In one possible implementation, a unique preset register address can be assigned to each computing unit, so that the queue manager can directly identify which computing unit issued the request by the written address.

[0056] In another implementation, multiple computing units can also share the same preset register address and distinguish their identities by writing data values ​​to that address.

[0057] Here, the write operation is not merely a simple data transfer; more importantly, it can directly trigger the queue manager to dispatch tasks. Alternatively, the hardware logic of the task dispatch system can be understood as follows: once a write operation to a preset register address is detected, an internal hardware event is generated, which immediately activates the queue manager's task dispatch logic. This mechanism is called "Doorbell Write," and its behavior is analogous to pressing a doorbell; the write operation itself is a notification signal, its purpose being to wake up or trigger subsequent processing flows.

[0058] This "doorbell write" method simplifies the process of the computing unit sending task requests into a single, low-overhead hardware write instruction. It completely bypasses traditional communication methods based on software interrupts or complex message passing protocols, thus avoiding the significant performance overhead and time delays caused by operating system intervention, context switching, and other operations.

[0059] In this embodiment of the invention, by specifying the sending method of the task request as a write operation to a preset register address, and using this write operation to directly trigger the queue manager at the hardware level, an ultra-low latency communication path from the computing unit to the scheduling center is constructed. This allows the computing unit to notify the queue manager in a near-instantaneous manner after completing the task, greatly shortening the connection window between tasks and further compressing the idle time of the computing unit, thereby further improving the response speed and efficiency of the entire dynamic scheduling system.

[0060] Based on the above embodiments, the system also includes a shared storage area; the shared storage area is accessible to the queue manager and the computing unit;

[0061] The queue manager is specifically used to write the identifier of the target task block into the storage location corresponding to the computing unit in the shared storage area;

[0062] The computing unit is further configured to read the identifier of the target task block from the storage location in the shared storage area and store the identifier of the target task block in the identifier location in the local cache; the identifier location is used to store the identifier of the task block to be processed by the corresponding computing unit.

[0063] Specifically, in order to achieve reliable and conflict-free transmission of the task block identifier tb id, this embodiment of the invention also introduces a key communication medium, namely a shared storage area.

[0064] In detail, the task dispatch system also includes a cluster shared memory (CSM). This shared memory can be a dedicated on-chip static random access memory or a specific area partitioned from the system's L2 cache (such as Level 2 Cache). Its core function is that it allows the queue manager and multiple computing units to jointly read and write to it. This shared memory acts like a central "bulletin board," building an efficient "data exchange bridge" between the previously independent central controller and the various computing units.

[0065] In this architecture, when the queue manager obtains the identifier of a target task block from the task queue, it writes the identifier to the storage location in the shared memory area corresponding to the computing unit that initiated the task request. It's important to note that each computing unit has its own dedicated storage location in the shared memory area, ensuring targeted and conflict-free communication. For example, each computing unit's hardware ID can be statically mapped to a specific address in the shared memory area. Thus, when the queue manager responds to a task request from computing unit 3 (e.g., cu3), it accurately writes the obtained target task block identifier to cu3's dedicated storage location, thereby avoiding confusion or data overlap in task assignments with other computing units.

[0066] Accordingly, after sending a task request, the computing unit actively and repeatedly reads the contents of its corresponding storage location from the shared storage area to check if a new task block identifier has been written. Once a valid target task block identifier is read, the computing unit stores the identifier of the target task block in the identifier location in its local cache. Here, the local cache refers to the storage space with extremely high access speed owned by the computing unit or its computing processing cluster (SPC, streaming processor cluster). The identifier location is the register or memory unit in the local cache used to store the identifier of the target task block currently to be processed.

[0067] Here, retrieving the identifier of the target task block from the shared storage area and storing it in the local cache helps improve overall computing performance. Specifically, during task execution, the computing unit may need to use the identifier of the target task block multiple times, for example, to calculate the addresses of different data blocks. Placing it in the local cache with extremely low access latency avoids accessing the relatively distant shared storage area, which may be subject to access contention, each time it is used, thereby significantly improving the efficiency of task execution.

[0068] In this embodiment of the invention, a shared storage area accessible to multiple parties is introduced, and each computing unit is allocated a dedicated storage location within this area, thereby establishing a clear, reliable, and conflict-free asynchronous communication protocol. The queue manager dispatches tasks through write operations, and the computing unit receives tasks through read operations and caches the obtained identifiers in the local cache, which greatly optimizes the internal data access efficiency during the task execution phase and ensures the robustness and high performance of the task handover process in the dynamic scheduling system.

[0069] Based on the above embodiments, the computing unit is further configured to execute a memory barrier instruction before reading the identifier of the target task block from the storage location in the shared storage area;

[0070] Memory barrier instructions are used to invalidate cached data at a specified location in the local cache of a computing unit, and to enable the computing unit to directly access the shared memory region when obtaining the identifier of a target task block.

[0071] Specifically, in this embodiment of the invention, a core technical challenge in the task dispatch system is how to ensure data consistency and visibility. In actual operation, without special mechanisms, a computing unit may, due to its internal caching and out-of-order execution optimization strategies, read an expired task identifier written by the queue manager but not yet effective in the entire task dispatch system, leading to scheduling failure.

[0072] To address this issue, this embodiment of the invention introduces a mandatory synchronization barrier. Specifically, the computing unit is further configured to execute a memory barrier instruction (mbar) before attempting to read the identifier of the target task block from the corresponding storage location in the shared memory region. This memory barrier instruction is a special hardware instruction used to enforce the order and visibility of memory operations. It plays a crucial role in the dynamic scheduling process, and its core function can be summarized in the following two aspects:

[0073] First, the memory barrier instruction is used to invalidate cached data at a specified location in the computing unit's local cache. As mentioned earlier, the computing unit caches the identifier of the previous task block or other historical data at the specified location in its local cache. Without a memory barrier, when the computing unit obtains the identifier of a new task block through polling or other methods, its internal optimization logic may directly read data from the local cache, resulting in it repeatedly reading old, invalid identifiers and failing to perceive changes that have occurred in the shared storage area. In this embodiment of the invention, executing the memory barrier instruction forcibly marks the cache entry as invalid, which is equivalent to clearing the computing unit's old memory of the identifier, forcing it to read data from a higher-level shared storage area.

[0074] Second, after invalidating the cached data at the identified location in the local cache, the memory barrier instruction further establishes a "barrier" that forces subsequent read operations to bypass the local cache of the computing unit and directly access the shared storage area, which serves as the central hub for information exchange. This ensures that the computing unit reads the latest target task block identifier written by the queue manager and visible globally, rather than an outdated identifier from its own cache.

[0075] In this embodiment of the invention, inserting a memory barrier instruction before reading shared memory can construct a robust data synchronization mechanism. This mechanism forces the computing unit to discard potentially outdated cached data and directly obtain the latest information from the shared storage area. This fundamentally solves the risk of data inconsistency that may occur in asynchronous communication between multi-processor cores, ensuring that the computing unit can accurately receive the latest target task block identifier assigned by the queue manager after each task request. This provides a key underlying hardware guarantee for the stable and reliable operation of the entire high-performance dynamic scheduling system.

[0076] Based on the above embodiments, the computing unit is further configured to poll the storage location in the shared storage area, detect whether the queue manager has written the identifier of the target task block into the storage location; if so, it reads the identifier of the target task block from the storage location and terminates the polling.

[0077] Specifically, after the computing unit sends a task request, it does not enter a passive sleep state to wait for an interrupt signal. Instead, it is configured to actively poll its dedicated storage location in the shared storage area. Here, polling is an active, cyclical checking behavior. In this embodiment, the computing unit enters a compact hardware execution loop, repeatedly reading the contents of its storage location within the loop. Compared to traditional software interrupt mechanisms, the active waiting strategy in this embodiment effectively avoids complex context switching and interrupt handling overhead. Moreover, since the computing unit is idle at this time, utilizing its execution cycle for polling is an effective reuse of computing resources.

[0078] In each polling cycle, the core task of the computing unit is to check whether the queue manager has written the identifier of a new, valid target task block to the specified storage location. This check can be achieved by comparing the read value with a preset invalid or no-task status value. For example, this storage location might be set to 0 or -1 when there are no new tasks. When the value read by the computing unit is no longer this preset invalid value, it means that the queue manager has assigned a new task. Accordingly, the value of this storage location can be reset after the computing unit reads it.

[0079] If so, meaning a valid update to the storage location is detected, it indicates that the conditions for task handover have been met. At this point, the computing unit reads the identifier of the new target task block from the storage location and uses it as the credential for the new task to be executed. Immediately afterwards, the computing unit terminates polling. Terminating polling signifies that the computing unit has successfully acquired the task, exited the active waiting loop, and is ready to execute the task.

[0080] In conjunction with the aforementioned embodiments, the complete task assignment process is as follows: After completing the previous task block, the computing unit sends a task request through "doorbell write". Then, under the protection of the memory barrier instruction, it begins to enter a polling loop, continuously detecting its exclusive storage location in the shared storage area. Once a new target task block identifier is detected, the identifier is immediately taken away, the polling stops, and the unit proceeds to execute the new computing task.

[0081] In this embodiment of the invention, the introduction of a polling mechanism can minimize the waiting delay during the task handover process, ensuring that the computing unit can be put into the new task processing flow as quickly as possible once it is idle. This greatly improves the efficiency of task switching and the response speed of the dynamic scheduling system, and provides a guarantee for maximizing the utilization of computing resources and the overall performance of the system.

[0082] Based on the above embodiments, the queue manager is also used to detect whether the managed task queue is empty; if so, it returns a task end identifier to the computing unit that issued the task request.

[0083] The task end flag is used to stop the computing unit from making task requests.

[0084] Specifically, this embodiment of the invention also provides a mechanism for terminating the entire computing workload. In short, any efficient task dispatch system requires not only an efficient execution process but also a clear and reliable termination procedure.

[0085] In detail, in addition to its core responsibilities of responding to task requests and assigning task blocks, the queue manager is also configured to perform a preliminary check each time a task request is received, namely, to check whether the task queue it manages is empty.

[0086] In one specific implementation, the task queue is typically managed by a "head pointer" and a "tail pointer." The head pointer points to the position of the identifier of the next pending task block, while the tail pointer points to the position of the identifier of the last pending task block in the queue. Therefore, detecting whether the task queue is empty can be efficiently implemented by checking whether the head pointer has caught up with or equaled the tail pointer. When the head and tail pointers meet, it means that all initially stored task blocks in the queue have been assigned.

[0087] If so, that is, when the queue manager detects that the task queue is indeed empty, its behavior will undergo a crucial change. At this point, the queue manager will no longer attempt to retrieve the task block identifiers from the managed task queues, but will instead return a task completion identifier to the computing unit that just issued the task request. This task completion identifier is a preset, special signal value that is distinct from the identifiers of any valid task blocks. For example, if the identifiers of valid task blocks are all positive integers, the task completion identifier can be set to -1, 0, or a specific maximum value. Its core purpose is to serve as a clear "work completed" signal, notifying the computing unit that the entire computing task has ended.

[0088] Correspondingly, on the computing unit side, when it reads a value from the shared storage area, it first determines whether the value is an identifier of a valid target task block or a task end identifier. The purpose of the task end identifier is to cause the computing unit that receives it to stop making task requests. Specifically, when the computing unit recognizes that it has obtained a task end identifier, it will understand that the entire task pool has been cleared and there is no need to continue requesting new tasks. At this time, the computing unit will break out of the loop of task request, task assignment, and task execution, no longer send task requests to the queue manager, and can transition to the final idle state or wait for the task assignment system to initiate a brand new computing task.

[0089] In this embodiment of the invention, the queue manager is given the ability to detect the queue status and return a task completion flag, and the computing unit is given the termination behavior after receiving the flag. This provides a complete and efficient hardware-level termination mechanism for the entire dynamic scheduling system, ensuring that the entire system can automatically and orderly stop working when all task blocks are completed. This avoids the computing unit from continuing to perform invalid polling and requests after the task is completed, greatly saving power consumption and constituting an indispensable closing link in the dynamic scheduling mechanism.

[0090] Based on the above embodiments, the queue manager is specifically used to manage the task queue through head pointers and tail pointers;

[0091] The head pointer indicates the position of the identifier of the next target task block in the task queue; the tail pointer indicates the position of the identifier of the last unprocessed task block in the task queue.

[0092] The queue manager is also used to update the head pointer after retrieving the identifier of the target task block to be processed from the managed task queue.

[0093] Specifically, in order to operate on the task queue with minimal hardware overhead and maximum speed, the queue manager is specifically configured to manage the task queue through a pair of pointers, namely the head pointer and the tail pointer. Figure 3This is an interactive example diagram of the task assignment process provided by the present invention, such as... Figure 3 As shown, the head pointer and tail pointer are dedicated hardware registers inside the queue manager, used to track the status of the task queue in real time.

[0094] The tail pointer serves to mark the boundary of the task pool. Before the entire workload begins execution, the system loads the identifiers of all pending task blocks into the task queue sequentially, and the tail pointer is set to point to the position of the identifier of the last pending task block in the task queue. Throughout the task dispatch process, the tail pointer typically remains unchanged; it acts like a finish line, defining the total workload of this computational task.

[0095] In contrast to the tail pointer, the head pointer acts as a dynamic dispatcher. It always indicates the position of the identifier of the next target task block that can be assigned in the task queue. In other words, the head pointer points to the forefront of all unassigned task blocks in the current task pool.

[0096] Based on this two-pointer management mechanism, the core process of task dispatch becomes extremely efficient. That is, when the queue manager receives a task request, it does not need to perform any search or complex query, but directly uses the value of the position indicated by the head pointer as the memory address to read the identifier of the next target task block to be processed from the task queue.

[0097] After successfully retrieving the identifier of the target task block to be processed from the position indicated by the head pointer, the queue manager updates the head pointer. This update operation increments the head pointer by one (or offsets it according to the queue's storage structure), making it point to the next position in the queue. Logically, updating the head pointer is equivalent to removing a task block from the task pool, thus ensuring that the same task block is not reassigned.

[0098] In this embodiment of the invention, the queue management mechanism based on head and tail pointers simplifies the complex logic management of the task queue into extremely fast pointer register read / write and arithmetic operations, greatly reducing the hardware overhead and processing latency of task assignment. Furthermore, this mechanism also greatly simplifies the determination of whether the task queue is empty; that is, it only requires comparing the values ​​of the head and tail pointers in the hardware, thus providing a solid guarantee for the rapid response and robust operation of the entire dynamic scheduling system.

[0099] Based on the above embodiments, the computing unit is further configured to determine the data storage address of the task data corresponding to the target task block based on the identifier of the target task block, read the task data from the data storage address, and perform task execution based on the task data.

[0100] Specifically, once the computing unit obtains the identifier of the target task block assigned by the queue manager, its work is not yet complete, because the identifier is merely a lightweight sign rather than the actual data to be processed. Therefore, after obtaining the identifier, the computing unit still needs to perform subsequent addressing and data loading operations based on the identifier.

[0101] In detail, the computing unit first uses this identifier to determine the exact location of the data to be processed in global memory, that is, the data storage address of the task data corresponding to the target task block. This process is essentially an address decoding or calculation process. Figure 4 This is an example diagram of the computing chip provided by the present invention, such as... Figure 4 As shown, in the specific implementation, the data for the entire computation task can be stored contiguously in a global memory, namely High Bandwidth Memory (HBM). The computation unit can calculate the exact data address based on the memory base address, the identifier of the target task block, and the preset task block size.

[0102] Here, the memory base address is the starting address of the data storage area for the entire computing task, and the preset task block size is a constant representing the amount of data space occupied by each task block. Through simple calculations, the computing unit can accurately locate the exact location of the data it needs to process in global memory.

[0103] After determining the data storage address, the computing unit initiates one or more memory read operations to retrieve task data from that address. This task data is the raw input necessary for performing the computational task; for example, certain rows or columns of a matrix, or a pixel block in an image. This data is retrieved from global memory and loaded into local registers or local shared memory within the computing unit for high-speed access.

[0104] Finally, once all the necessary task data is ready, the computing unit will execute the task based on this loaded task data. This is the core computing step in the entire process. The computing resources inside the computing unit, such as the arithmetic logic unit, will begin to execute the specific operations defined in the task block, such as matrix multiplication, convolution, and vector addition.

[0105] In this embodiment of the invention, by giving the computing unit the ability to autonomously calculate data addresses based on identifiers, the decentralization and automation of task execution are realized. After obtaining the task identifier, the computing unit can immediately and independently complete the entire process of data location, data loading and final calculation, thereby efficiently converting the instructions generated by the system into actual computing power. This ensures that the entire dynamic scheduling mechanism can smoothly drive the actual computing tasks and minimizes the delay from task allocation to task execution.

[0106] Based on the above embodiments, the computing unit is further configured to determine the global task block identifier based on the identifier of the target task block and the offset of the computing unit in its respective computing unit group; and determine the data storage address based on the global task block identifier.

[0107] Specifically, when assigning tasks, the queue manager does not assign task blocks to each computing unit individually, but rather assigns them as a group of computing units. This reduces the complexity and overhead of scheduling. Therefore, after obtaining the identifier, the computing unit still needs to convert the identifier into the specific task it needs to execute.

[0108] Therefore, in this embodiment of the invention, the computing unit is further configured to determine a global threadblock ID, i.e., a global task block identifier, when calculating the data storage address. Specifically, this can be achieved by calculating the global task block identifier based on the identifier of the target task block and the offset (cu offsets in cluster) of the computing unit within its respective computing unit group, which is fixed during hardware design.

[0109] The offset here refers to a unique, static local ID of a computation unit within its computation unit group. For example, if a computation unit group contains 4 computation units, the offsets of these 4 computation units can be set to 0, 1, 2, and 3 respectively. This offset allows each computation unit to know its relative position within the group.

[0110] Specifically, here, the computing unit can directly add the identifier of the target task block to its own offset to obtain a unique global task block identifier. Then, the computing unit will use this global task block identifier to ultimately determine the data storage address. For example, the data storage address can be obtained by adding the product of the global task block identifier and the preset task block size to the memory base address.

[0111] In this embodiment of the invention, the global task block identifier is obtained through offset calculation, and the final data storage address is calculated based on this global task identifier. This greatly reduces the scheduling burden of the queue manager, enabling it to manage larger-granularity computing unit groups, thereby significantly improving the scalability of the entire system. Simultaneously, fine-grained task allocation is delegated to each computing unit group, allowing each unit to complete the task autonomously, achieving decentralized management and making the system architecture more flexible and efficient.

[0112] Furthermore, it is worth mentioning that, in addition to implementing dynamic scheduling through dedicated hardware logic as described above, the dynamic scheduling mechanism in this embodiment of the invention can also be implemented at the software level to achieve a similar dynamic load balancing effect without modifying the underlying hardware logic.

[0113] Specifically, in the software implementation, the core idea is also to break the static binding relationship between computing tasks and computing units. Instead of pre-allocating fixed task blocks to each computing unit, a task pool shared by all computing units is created, allowing idle computing units to actively draw numbers from it to claim new tasks.

[0114] To achieve this goal, this embodiment of the invention employs a Persistent Kernel programming model. Specifically, when a computing task (Kernel) starts, it launches a thread pool consisting of multiple thread blocks (also known as CTAs in Compute Unified Device Architecture, which can be understood as computing units in hardware) all at once. The key characteristic of these thread blocks is their persistence; that is, after completing a work unit (workload, which can be understood as a task block in hardware), they do not immediately exit but continue to "live" and actively request the next work unit in a loop until the entire task pool is exhausted.

[0115] The software implementation process is as follows:

[0116] At the software level, a global atomic counter, or global task index pointer (ptr), is used to replace the hardware queue manager and is set in global memory. This counter is a single integer variable, usually initialized to 0. It acts as a central task dispatcher, and its value represents the total number of task blocks that have been dispatched, indirectly pointing to the next task block to be processed.

[0117] When a persistent threadblock is ready to receive a new task, for example, at the beginning of kernel startup or after completing the previous unit of work, the most critical step in the whole process is executed: an atomic operation is performed on the global atomic counter, usually atomically incremented by one (atomicAdd or atomicInc).

[0118] Atomic operations are the fundamental means of ensuring data consistency in a multi-threaded environment. When hundreds or thousands of thread blocks may attempt to retrieve a number at the same time, atomic operations ensure that the reading, modification, and writing processes of the global atomic counter are an indivisible whole, thereby eliminating race conditions where multiple thread blocks obtain the same task ID.

[0119] This atomic operation typically returns a value. In this embodiment of the invention, it can return the counter value before the addition operation was performed. Therefore, after a thread block performs an atomic operation, it obtains a unique tile id assigned to its own task block.

[0120] After successfully obtaining a unique task_id through an atomic operation, the thread block will first check the exit condition. That is, it will compare the task_id with the pre-stored total number of tasks (tile count). If the task_id is greater than or equal to the total number of tasks, it means that the task pool has been exhausted, and the thread block will then break out of the work loop and exit safely.

[0121] If task_id is less than the total number of tasks, the thread block will, as in the hardware embodiment, calculate the specific location of the task data in global memory based on the identifier (e.g., parse the corresponding matrix row and column coordinates based on the tile id), then load the data and perform the calculation.

[0122] After the computation of the task block is completed, due to its persistent nature, the thread block will not terminate, but will perform atomic operations on the global atomic counter again to request the next task block.

[0123] In this embodiment of the invention, by employing a persistent kernel programming model and utilizing an atomic counter in global memory as a generator, a dynamic scheduling mechanism that actively pulls tasks from the task pool is reproduced at the software level. Whenever a thread block completes a task, it obtains the identifier of a new task block through an atomic operation. This achieves the dynamic effect of "assigning the first available task block to the idle thread block," effectively avoiding hardware idleness and performance "tailing" problems caused by uneven computation time of different task blocks. It achieves the goal of "fully utilizing" all computing units as much as possible, significantly improving overall performance. Compared to hardware implementation, this software solution has higher flexibility and greater versatility, and can be deployed on any parallel computing hardware that supports atomic operations.

[0124] The present invention also provides a computing chip, Figure 5 This is a schematic diagram of the computing chip provided by the present invention, as shown below. Figure 5 As shown, the chip includes a task dispatch system 510 as described in any of the preceding claims.

[0125] Specifically, the computing chip in this embodiment of the invention can be an application-specific integrated circuit (ASIC) designed to perform massively parallel computing, such as a GPU, an AI accelerator, an ASIC, or a programmable logic device configured to perform specific functions. At the physical level, the chip integrates massive computing resources (i.e., multiple computing units), a multi-level storage system (including local cache, shared storage areas, and global memory), and complex control logic on its silicon wafer.

[0126] In this embodiment of the invention, the core advantage of the computing chip lies in its deep integration of the task dispatch system described in any of the foregoing embodiments at the hardware architecture level. This means that the task dispatch system is not a software program running on a general-purpose processor, but rather is embedded as a dedicated hardware circuit and logic within the chip.

[0127] The central controller's queue manager is implemented as one or more dedicated hardware logic units within the chip's central control module. Multiple computing units refer to the hundreds or thousands of physical units on the chip used to perform actual calculations. Task queues and shared memory areas correspond to specific high-speed on-chip memories allocated on the chip.

[0128] The entire dynamic scheduling process, including "doorbell write" triggering, memory barrier synchronization, pointer management, etc., is directly executed by the chip's internal hardware state machine and control path.

[0129] By integrating the task assignment system as native hardware, the computing chip in this embodiment of the invention gains a fundamental performance advantage over traditional chips. It completely abandons the rigid static task allocation mode at the hardware level and turns to an efficient dynamic scheduling mechanism. This enables the computing chip to balance the load between all computing units in real time and dynamically when running complex workloads with uneven task durations, fundamentally eliminating the idle and wasted computing resources caused by the "tail effect".

[0130] The computing chip provided by this invention, due to its embedded high-efficiency hardware dynamic task allocation system, can more efficiently tap into the parallel processing potential of all computing units on the chip. Ultimately, this directly translates into an improvement in the chip's overall throughput and energy efficiency, giving it a stronger advantage in high-performance computing fields such as artificial intelligence training and inference, scientific computing, and graphics rendering.

[0131] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0132] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A task assignment system, characterized in that, It includes a central controller and multiple computing units; the central controller includes a queue manager, which manages the task queues in the central controller, and the task queues store the identifiers of multiple task blocks to be processed; The computing unit is used to send a task request to the queue manager upon completion of any task block; The queue manager is used to respond to the task request, obtain the identifier of the target task block to be processed from the managed task queue, and return the identifier of the target task block to the computing unit corresponding to the task request. The computing unit is also used to receive the identifier of the target task block and perform task execution based on the identifier of the target task block; It also includes a shared storage area; the shared storage area is accessible to the queue manager and the computing unit. The queue manager is specifically used to write the identifier of the target task block into the storage location corresponding to the computing unit in the shared storage area; The computing unit is further configured to read the identifier of the target task block from the storage location in the shared storage area, and store the identifier of the target task block in the identifier location in the local cache; the identifier location is used to store the identifier of the task block to be processed by the corresponding computing unit.

2. The task assignment system according to claim 1, characterized in that, The computing unit is specifically used to send the task request by performing a write operation to a preset register address in the central controller when any task block is completed. The write operation is used to trigger the queue manager to assign tasks.

3. The task assignment system according to claim 1, characterized in that, The computing unit is further configured to execute a memory barrier instruction before reading the identifier of the target task block from the storage location in the shared storage area; The memory barrier instruction is used to invalidate the cached data at the identified location in the local cache of the computing unit, and to enable the computing unit to directly access the shared storage area when obtaining the identifier of the target task block.

4. The task assignment system according to claim 1, characterized in that, The computing unit is further configured to poll the storage location in the shared storage area, detect whether the queue manager has written the identifier of the target task block into the storage location; if so, read the identifier of the target task block from the storage location and terminate the polling.

5. The task assignment system according to any one of claims 1 to 4, characterized in that, The queue manager is also used to detect whether the managed task queue is empty; if so, it returns a task end identifier to the computing unit that issued the task request. The task end flag is used to stop the computing unit from making task requests.

6. The task assignment system according to any one of claims 1 to 4, characterized in that, The queue manager is specifically used to manage the task queue through head and tail pointers; The head pointer indicates the position of the identifier of the next target task block in the task queue; the tail pointer indicates the position of the identifier of the last unprocessed task block in the task queue. The queue manager is also used to update the head pointer after obtaining the identifier of the target task block to be processed from the managed task queue.

7. The task assignment system according to any one of claims 1 to 4, characterized in that, The computing unit is further configured to determine the data storage address of the task data corresponding to the target task block based on the identifier of the target task block, read the task data from the data storage address, and perform task execution based on the task data.

8. The task assignment system according to claim 7, characterized in that, The computing unit is further configured to determine the global task block identifier based on the identifier of the target task block and the offset of the computing unit in its respective computing unit group; and to determine the data storage address based on the global task block identifier.

9. A computing chip, characterized in that, Includes the task assignment system as described in any one of claims 1 to 8.

Images