Task diagram initiation methods, devices, equipment, media and products

By generating task description information and configuring chained scheduling information for computing nodes in the task graph, the problem of high task graph startup overhead is solved, and a more efficient task graph startup process is achieved.

CN123114693APending Publication Date: 2026-11-17KUNWANG (SHANGHAI) TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611452144.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-17
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In parallel computing devices, the startup overhead and command submission overhead of the task graph increase with the number of computing nodes, leading to excessive host processing overhead and affecting the efficiency of the task graph.

Method used

During the instantiation or update phase of the task graph, task description information is generated for each computing node, and chained scheduling information is configured in it. The chained scheduling information is used to indicate the task description information of the successor computing nodes of the computing node. When the task graph is started, the task description information of the entry computing node is submitted, so that the device scheduler can schedule the successor computing nodes according to the chained scheduling information.

Benefits of technology

This reduces the processing overhead and command submission overhead of the host when starting the task graph each time, improves the startup efficiency of the task graph, and reduces the occupation and waiting time of the device task queue.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123114693A_ABST
    Figure CN123114693A_ABST
Patent Text Reader

Abstract

This disclosure provides a task graph initiation method, apparatus, device, medium, and product. It relates to the field of computer technology, specifically to task scheduling technology for parallel computing devices. The specific implementation includes: generating corresponding task description information for each computing node in the task graph during the instantiation or update phase of the task graph; configuring chained scheduling information in the task description information based on the dependencies between computing nodes, wherein the chained scheduling information at least indicates the task description information of the successor computing nodes of the current computing node; and, when initiating the task graph, submitting the task description information of the entry computing node of the task graph, so that after the computing node completes execution, the device scheduler schedules the task description information of the successor computing nodes of that computing node based on the chained scheduling information in the task description information of that computing node.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to task scheduling technology for parallel computing devices, specifically to a task graph initiation method, apparatus, electronic device, non-transitory computer-readable storage medium, and computer program product. Background Technology

[0002] In parallel computing devices such as general-purpose graphics processing units (GPGPUs), the task graph mechanism can pre-capture multiple computational tasks and their dependencies into a task graph, which can then be reused in subsequent executions. However, during task graph startup, the host still needs to traverse each compute node in the task graph and submit startup commands for each compute node to the device based on their dependencies. As the number of compute nodes in the task graph increases, the processing overhead and command submission overhead for the host increase each time the task graph is started.

[0003] Therefore, how to reduce the startup overhead of the task graph becomes an issue that needs attention. Summary of the Invention

[0004] This disclosure provides a task graph initiation method, apparatus, device, medium, and product.

[0005] According to one aspect of this disclosure, a task graph initiation method is provided, comprising: generating corresponding task description information for each computing node in the task graph during the instantiation or update phase of the task graph; configuring chained scheduling information in the task description information according to the dependencies between the computing nodes, wherein the chained scheduling information is used at least to indicate the task description information of the successor computing nodes of the computing node; and, when initiating the task graph, submitting the task description information of the entry computing node of the task graph, so that after the computing node completes execution, the device scheduler schedules the task description information of the successor computing nodes of the computing node according to the chained scheduling information in the task description information of the computing node.

[0006] According to another aspect of this disclosure, a task graph initiation apparatus is provided, comprising: a generation module, configured to generate corresponding task description information for each computing node in the task graph during the instantiation or update phase of the task graph; a configuration module, configured to configure chained scheduling information in the task description information according to the dependencies between the computing nodes, wherein the chained scheduling information is used at least to indicate the task description information of the successor computing nodes of the computing node; and a submission module, configured to submit the task description information of the entry computing node of the task graph when initiating the task graph, so that after the computing node completes execution, the device scheduler schedules the task description information of the successor computing nodes of the computing node according to the chained scheduling information in the task description information of the computing node.

[0007] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the task graph initiation method as described above.

[0008] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause a computer to perform the task graph initiation method described above.

[0009] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the task graph initiation method described above.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 This is a flowchart illustrating a task graph initiation method provided in an embodiment of this disclosure; Figure 2 This is a flowchart illustrating another task graph initiation method provided in this embodiment of the disclosure; Figure 3 This is a flowchart illustrating another task graph initiation method provided in this embodiment of the disclosure; Figure 4 This is a flowchart illustrating another task graph initiation method provided in this embodiment of the disclosure; Figure 5 This is a flowchart illustrating another task graph initiation method provided in this embodiment of the disclosure; Figure 6 This is an exemplary schematic diagram of the dependencies between computing nodes in the task graph provided in this embodiment of the disclosure; Figure 7 This is a schematic diagram of a task graph initiation device provided in an embodiment of this disclosure; Figure 8 This is a block diagram of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0012] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0013] The terms such as "first," "second," and "third" used herein are used only to distinguish one entity (or operation) from another, and are not intended to require or imply a specific order or relationship between these entities (or operations). Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0014] The collection, storage, use, processing, transmission, provision, and disclosure of any type of information, such as user personal information, in this technical solution comply with relevant laws and regulations and do not violate public order and good morals.

[0015] Before describing the technical solutions provided by the embodiments of this disclosure, in order to facilitate the understanding of the embodiments of this disclosure, this disclosure first specifically explains the problems existing in the related technologies.

[0016] In parallel computing devices such as general-purpose graphics processors, the task graph mechanism can pre-capture multiple computing tasks and their dependencies into a task graph, and reuse the task graph in subsequent execution through a single startup operation, thereby reducing the overhead of repeatedly constructing task topologies.

[0017] A common implementation involves performing topology checks, resource analysis, and node metadata caching during the task graph instantiation phase. When starting the task graph, the host traverses each compute node according to their dependencies and submits the startup command for each compute node to the device sequentially. As the number of compute nodes in the task graph increases, the processing overhead for the host increases each time the task graph is started; a large number of startup commands also occupy the device's task queue, causing the host to wait for the device's task queue to be emptied.

[0018] Another approach is to cache the static startup configuration of compute nodes during the task graph instantiation phase, update the changed parameters when starting the task graph, and then commit each compute node individually. This approach can reduce the overhead of redundant configuration, but the host still needs to process and commit each compute node every time the task graph is started, and the startup overhead remains large when the number of compute nodes is large.

[0019] In view of this, this disclosure provides a task graph startup method aimed at reducing the startup overhead of task graphs.

[0020] This disclosure provides a task graph initiation method, referring to... Figure 1 This includes the following steps: Step 101: In the instantiation or update phase of the task graph, generate corresponding task description information for each computing node in the task graph. Step 102: Configure chained scheduling information in the task description information according to the dependency relationship between each computing node. The chained scheduling information is used at least to indicate the task description information of the successor computing node of the computing node. Step 103: When starting a task graph, submit the task description information of the entry computing node of the task graph so that after the computing node finishes execution, the device scheduler can schedule the task description information of the successor computing node according to the chain scheduling information in the task description information of the computing node.

[0021] The following is a detailed description with reference to specific examples.

[0022] In step 101, during the instantiation or update phase of the task graph, corresponding task description information is generated for each computing node in the task graph.

[0023] In step 101, the CUDA Graph includes multiple compute nodes and the dependencies between them. A compute node is a task node in the graph that needs to be executed by a parallel computing device, and the dependencies between compute nodes are used to indicate the execution order of each compute node.

[0024] For example, if compute node B needs to wait for compute node A to finish executing before it can execute, then compute node A is the predecessor compute node of compute node B, and compute node B is the successor compute node of compute node A.

[0025] The Task Description Information (DESC) is descriptive data corresponding to a compute node, used by the device scheduler to start the corresponding compute node. The Task Description Information may carry the data required to start the corresponding compute node, and the specific content included in the Task Description Information can be determined according to the startup requirements of the corresponding compute node. This embodiment of the disclosure does not impose any limitations on this.

[0026] The graph instantiation phase refers to the stage where the executable form of the task graph is generated based on its topology and the information of each compute node. The graph update phase refers to the stage where the instantiated task graph is modified. During the instantiation or update phases, task description information corresponding to each compute node is generated, allowing the previously generated task description information to be reused when the task graph is launched subsequently, eliminating the need to regenerate all task description information each time.

[0027] In step 102, based on the dependencies between computing nodes, chained scheduling information is configured in the task description information. The chained scheduling information is used to indicate the task description information of the successor computing nodes of the computing nodes.

[0028] In step 102, the chained scheduling information is a dependent DESC-related field configured in the task description information corresponding to the compute node. This establishes the association between the task description information of the compute node and its successor compute nodes, thus forming a DESC chaining launch mechanism on the device side. For any compute node with successor compute nodes, the chained scheduling information in its task description information at least indicates the task description information corresponding to that successor compute node. After the compute node completes its execution, the device scheduler can determine the successor compute node that needs to be scheduled based on this chained scheduling information.

[0029] Chained scheduling information can be represented in any data format that the device scheduler can recognize, as long as it enables the device scheduler to determine the task description information of its successor computing node based on the task description information of a computing node. This disclosure does not limit the specific fields, encoding format and storage method of chained scheduling information.

[0030] Since the chained scheduling information is configured during the instantiation or update phase of the task graph, the configured chained scheduling information can be reused when the task graph remains unchanged, without the host needing to re-analyze the dependencies between computing nodes each time the task graph is started.

[0031] In step 103, when a task graph is started, the task description information of the entry computing node of the task graph is submitted so that after the computing node finishes execution, the device scheduler can schedule the task description information of the successor computing node according to the chain scheduling information in the task description information of the computing node.

[0032] In step 103, the root node is the first computing node to be triggered when the task graph is started. After the host submits the task description information of the root node to the device scheduler, the device scheduler starts the root node according to the task description information.

[0033] After any computing node completes its execution, the device scheduler determines the task description information of the successor computing node based on the chained scheduling information in the task description information of that computing node, and schedules the task description information of the successor computing node, causing the corresponding successor computing node to enter the execution process. After the successor computing node completes its execution, the device scheduler continues to schedule the next computing node based on the chained scheduling information of that successor computing node, until all computing nodes that need to be executed in the task graph have completed their execution.

[0034] For example, the task graph includes computing node A, computing node B, and computing node C, where computing node A is the predecessor computing node of computing node B, and computing node B is the predecessor computing node of computing node C. During the instantiation phase of the task graph, task description information S1 is generated for computing node A, task description information S2 is generated for computing node B, and task description information S3 is generated for computing node C. The task description information S1 is configured with chained scheduling information indicating task description information S2, and the task description information S2 is configured with chained scheduling information indicating task description information S3.

[0035] When the task graph is initiated, the host submits task description information S1 corresponding to the entry computing node A. After computing node A completes its execution, the device scheduler schedules task description information S2 according to the chained scheduling information in task description information S1, causing computing node B to execute. After computing node B completes its execution, the device scheduler schedules task description information S3 according to the chained scheduling information in task description information S2, causing computing node C to execute. During this process, the host does not need to submit task description information for computing nodes B and C separately.

[0036] Based on this, during the instantiation or update phase of the task graph, the host configures chained scheduling information in the task description information according to the dependencies between computing nodes. When starting the task graph, the host submits the task description information of the entry computing node, and subsequent computing nodes are scheduled sequentially by the device scheduler according to the chained scheduling information. Thus, the host does not need to traverse each computing node and submit a start command for each node every time the task graph is started, reducing the host's processing overhead and command submission overhead, and lowering the startup overhead of the task graph.

[0037] According to some embodiments of this disclosure, in order for the device scheduler to determine the successor computing node and its processing method, the above-mentioned chained scheduling information may optionally include at least dependent DESC pointer and dependent DESC action.

[0038] Among them, the successor indication information is used to indicate the storage address of the task description information of the successor computing node on the device side, and the dependency action information is used to indicate the hardware action that the device scheduler will perform on the task description information of the successor computing node after the computing node is completed.

[0039] Specifically, during the instantiation or update phase of the task graph, the storage location of the task description information of the successor computing node on the device side is determined based on the dependencies between the computing nodes, and the corresponding storage address is written into the successor indication information in the task description information of the predecessor computing node. Simultaneously, based on the processing required by the task description information of the successor computing node, the corresponding dependency action information is configured.

[0040] For example, the storage location mentioned above can be a device virtual address.

[0041] After a computing node completes its execution, the device scheduler accesses the task description information of the successor computing node based on the successor indication information in the task description information of that computing node, and performs the corresponding operation on the task description information of the successor computing node according to the operation type indicated by the dependent action information.

[0042] For example, computing node A is the predecessor computing node of computing node B. Computing node A corresponds to task description information S1, and computing node B corresponds to task description information S2. The successor indication information in task description information S1 records the storage address of task description information S2 on the device side, and the dependent action information records the type of operation that needs to be performed on task description information S2 after computing node A completes its operation. After computing node A completes its operation, the device scheduler retrieves task description information S2 based on the storage address and processes task description information S2 according to the dependent action information.

[0043] In this disclosure, the chained scheduling information carries the storage address of the task description information of the successor computing node and the corresponding operation type. After the computing node is completed, the device scheduler can directly determine the successor computing node to be processed and its processing method. The host does not need to provide the address and processing command of the successor computing node one by one during the task graph startup phase, which can reduce the host processing overhead and speed up the progress between adjacent computing nodes.

[0044] According to some embodiments of this disclosure, in order for the device scheduler to identify valid subsequent indication information, the chained scheduling information may optionally include an enable flag. Step 102 may also include: configuring an enable flag (dependent DESC enable) in the task description information, which corresponds one-to-one with the subsequent indication information and is used to indicate whether the corresponding subsequent indication information is valid.

[0045] Specifically, the task description information may include one or more successor indication information, each of which corresponds to an enable flag. For successor indication information whose storage address for the task description information of a configured successor computing node is enabled, the corresponding enable flag is configured to be valid; for successor indication information whose storage address is not configured to be valid, the corresponding enable flag is configured to be invalid.

[0046] In one implementation, each subsequent indication information is configured in the dependent DESC slot of the task description information. The enable flag corresponds one-to-one with the dependent DESC slot, and the device scheduler determines the dependent DESC slot to be accessed based on the enable flag.

[0047] After the computing node completes its execution, the device scheduler checks the corresponding subsequent indication information based on each enable flag: for subsequent indication information with a valid enable flag, it accesses the storage address indicated by the subsequent indication information; for subsequent indication information with an invalid enable flag, it skips the subsequent indication information and does not access its corresponding storage address.

[0048] For example, computing node A has a successor computing node B, and task description information S1 includes two successor indication information items. The first successor indication information records the storage address of task description information S2, and its corresponding enable flag is valid; the second successor indication information is not configured with a valid address, and its corresponding enable flag is invalid. After computing node A completes execution, the device scheduler accesses the first successor indication information and skips the second successor indication information.

[0049] In this disclosure, an enable flag is used to distinguish between valid and invalid successor indication information. The device scheduler does not need to access successor indication information without a configured valid address, thus reducing invalid access. At the same time, the same task description information structure can be compatible with scenarios with no successor computing node, with one successor computing node, and with multiple successor computing nodes, improving the applicability of the task description information structure.

[0050] According to some embodiments of this disclosure, in order to adapt to the different processing requirements of subsequent computing nodes, the above hardware actions may optionally consist of at least one of the following operation types: scheduling operation, copy operation, invalidate operation, and decrement dependence operation.

[0051] The scheduling operation is used to schedule the task description information of subsequent computing nodes, enabling the corresponding subsequent computing nodes to enter the executable process. After a computing node completes its execution, if its dependent action information indicates a scheduling operation, the device scheduler obtains the task description information of the subsequent computing node based on the subsequent indication information and schedules that task description information.

[0052] The copy operation is used to load the task description information of the successor compute node into the device-side scheduling cache (DESCcache). The device-side scheduling cache is a cache used by the device scheduler to store loaded task description information. By using the copy operation, the task description information of the successor compute node can be made accessible on the device side before the scheduling operation is triggered.

[0053] The invalidation operation is used to invalidate old task description information already loaded in the device-side scheduling cache. After the device scheduler executes the invalidation operation, the old task description information is no longer considered a valid scheduling object to avoid subsequent use of the old task description information.

[0054] The decrement operation is used to decrement the dependency counter value in the task description information of the successor compute node by one. The dependency counter value records the number of predecessor compute nodes that have not yet completed execution. Each time the device scheduler performs a decrement operation, it indicates that one of the predecessor compute nodes of the successor compute node has completed execution.

[0055] The above operation types can be implemented through the Launch Compare And Swap (LCAS) operation, which is an atomic operation sent to the device scheduler during the task graph startup process. The device scheduler can perform scheduling, copying, invalidation, or decrementing operations on the task description information of subsequent computing nodes based on dependency action information.

[0056] For example, if the dependency action information in task description information S1 indicates a scheduling operation, then after computing node A completes its execution, the device scheduler schedules task description information S2; if the dependency action information indicates a copy operation, then after computing node A completes its execution, the device scheduler loads task description information S2 into the device-side scheduling cache, but does not schedule task description information S2; if the dependency action information indicates an invalidation operation, then the device scheduler invalidates the old task description information S2 already loaded in the device-side scheduling cache; if the dependency action information indicates a decrement operation, then the device scheduler decrements the dependency count value in task description information S2 by one.

[0057] In this disclosure, the dependency action information can distinguish between different processing methods, such as direct scheduling, loading into the scheduling cache, invalidating old task description information, and updating dependency count values. The device scheduler can execute the corresponding operation according to the dependency action information configured in the task description information, without the host needing to specify the processing method node by node during the task graph startup phase. This improves the flexibility of chained scheduling and reduces host processing overhead.

[0058] According to some embodiments of this disclosure, in order to express the processing methods of different dependent edges through dependent action information, optionally, the dependent action information configured in step 102 indicates any one of the following four hardware actions: a first hardware action, a second hardware action, a third hardware action, and a fourth hardware action.

[0059] The first hardware action consists of a scheduling operation, used to directly schedule the task description information of the successor computing node. For dependency edges where the task description information of the successor computing node has already been loaded onto the device side and does not need to be updated, the device scheduler can directly schedule the task description information of the successor computing node after the predecessor computing node has completed its execution.

[0060] The second hardware action consists of a copy operation and a scheduling operation executed sequentially. The device scheduler first loads the task description information of the successor computing node into the device-side scheduling cache, and then schedules the loaded task description information. The second hardware action is suitable for scenarios where the task description information of the successor computing node needs to be loaded into the device-side scheduling cache before entering the scheduling process.

[0061] The third hardware action consists of a failure operation, a copy operation, and a scheduling operation executed sequentially. The device scheduler first invalidates the old task description information already loaded in the device-side scheduling cache, then loads the updated task description information of the successor compute node into the device-side scheduling cache, and finally schedules the updated task description information. This third hardware action is suitable for scenarios where the task description information of the successor compute node is updated before this startup, and can avoid the device scheduler scheduling the old task description information in the device-side scheduling cache.

[0062] The fourth hardware action consists of a decrement operation, used to decrement the dependency count in the task description information of the successor computing node. For a successor computing node with multiple predecessor computing nodes, after each predecessor computing node completes its execution, the device scheduler decrements the dependency count of the successor computing node by one; when the dependency count decreases to zero, the device scheduler schedules the task description information of that successor computing node.

[0063] For example, compute node A is a predecessor compute node of compute node B. If task description information S2 is already in a schedulable state, the dependency action information in task description information S1 can indicate a first hardware action, and compute node A will directly schedule task description information S2 after execution. If task description information S2 needs to be loaded into the device-side scheduling cache first, the dependency action information can indicate a second hardware action. If task description information S2 is updated before this startup, the dependency action information can indicate a third hardware action. If compute node B has other predecessor compute nodes, the dependency action information can indicate a fourth hardware action to decrement the dependency count value in task description information S2.

[0064] In this disclosure, dependency action information can indicate four hardware actions: direct scheduling, post-load scheduling, post-update scheduling, and dependency count decrementing, thereby configuring the processing semantics of different dependency edges into the task description information. After the predecessor computing node completes, the device scheduler can execute the corresponding hardware actions on the task description information of the successor computing node based on the dependency action information. This eliminates the need for the host to determine the processing method of each dependency edge every time the task graph is started, thus reducing host processing overhead.

[0065] According to some embodiments of this disclosure, in order to handle a multi-predecessor join (DESCjoin) scenario with multiple predecessor computing nodes, optionally, for a target computing node with at least two predecessor computing nodes, the chained scheduling information further includes a dependency count value, and step 102 above may further include: Initialize the dependency count in the task description information of the target computing node to the number of predecessor computing nodes of the target computing node; configure the successor indication information pointing to the task description information of the target computing node in the task description information of each predecessor computing node; configure the dependency action information in the task description information of each predecessor computing node to perform a decrement operation on the dependency count value of the target computing node, so that the device scheduler schedules the task description information of the target computing node when the dependency count value of the target computing node is decremented to zero.

[0066] Specifically, during the instantiation or update phase of the task graph, the number of predecessor compute nodes of the target compute node is counted, and this number is written into the dependency count value in the task description information of the target compute node. The initial value of the dependency count value represents the number of predecessor compute nodes that the target compute node needs to wait for to complete execution.

[0067] For each predecessor compute node of the target compute node, a successor indication is configured in the task description information of the predecessor compute node, pointing it to the task description information of the target compute node, and the dependency action information is configured as a decrement operation. After any predecessor compute node completes its execution, the device scheduler accesses the task description information of the target compute node according to the successor indication information of the predecessor compute node, and performs a decrement operation on the dependency count value of the target compute node.

[0068] In one implementation, the device scheduler performs an atomic decrement operation on the dependency count, checking if the dependency count is zero after each decrement. If the dependency count is not zero, it indicates that the target compute node still has predecessor compute nodes that have not yet completed; if the dependency count decreases to zero, it indicates that all predecessor compute nodes of the target compute node have completed, and the device scheduler schedules the task description information of the target compute node. This atomic decrement operation avoids missing decrement operations when multiple predecessor compute nodes complete simultaneously, ensuring the accuracy of the dependency count update.

[0069] For example, both compute node A and compute node B are predecessor compute nodes of compute node C. The dependency count in the task description information S3 corresponding to compute node C is initialized to 2. Successor indication information pointing to task description information S3 is configured in task description information S1 and task description information S2 respectively, and the dependency action information of both is configured as a decrement operation. After compute node A completes execution, the device scheduler decrements the dependency count in task description information S3 from 2 to 1; after compute node B completes execution, the device scheduler decrements the dependency count from 1 to 0 and schedules task description information S3, enabling compute node C to enter the executable process.

[0070] In this disclosure, the dependency count of the target compute node records the number of its predecessor compute nodes that have not yet completed. After each predecessor compute node completes its task, it decrements the dependency count. The device scheduler schedules the target compute node when the dependency count reaches zero. Therefore, the merging relationship between multiple predecessor compute nodes is handled by the device scheduler based on the task description information. This eliminates the need for the host to wait for each predecessor compute node to complete before submitting the target compute node, reducing host processing overhead and preventing the target compute node from being scheduled prematurely or repeatedly.

[0071] In one implementation, the task graph initiation method provided in this disclosure can be applied to a computing system, which may include a host and a target device. The target device can be a parallel computing device for executing the task graph, such as a general-purpose graphics processor or an artificial intelligence acceleration chip. The device scheduler is a scheduling unit or scheduling module set in the target device. That is, the host submits the task description information of the entry computing node to the target device, and the device scheduler in the target device advances subsequent computing nodes according to the chained scheduling information.

[0072] According to some embodiments of this disclosure, in order to handle scenarios where one computing node corresponds to multiple subsequent computing nodes, optionally, step 102 above, configuring chained scheduling information in the task description information, may specifically include: In response to the number of successor computing nodes of the target computing node exceeding the maximum number of successor indications that can be configured for a single task description, at least one auxiliary task description (null DESC) is generated. The auxiliary task description does not correspond to any computing node in the task graph. Configure at least one auxiliary task description information in the chained scheduling information of the task description information of the target computing node; The task description information of at least some of the successor computing nodes of the target computing node is configured in the chained scheduling information of each auxiliary task description information, so that the device scheduler schedules each successor computing node of the target computing node according to the chained scheduling information of the auxiliary task description information.

[0073] Here, the upper limit of the number of configurable successor indications for a single task description is determined by the hardware architecture of the target device. This upper limit corresponds to the maximum fan-out number (max DESC Fork Degree) supported by the target device. When the number of successor computing nodes of the target computing node does not exceed this upper limit, the task description information of each successor computing node can be directly configured in the task description information of the target computing node. When the number of successor computing nodes of the target computing node exceeds this upper limit, the task description information of the target computing node cannot directly configure the task description information of all successor computing nodes. In this case, one or more auxiliary task description information is generated.

[0074] Auxiliary task description information is used to assist in scheduling and does not correspond to any computing node in the task graph. Auxiliary task description information corresponds to null operations, which do not execute actual computing tasks, but have the configuration capability of chained scheduling information and can be used to take over some of the successor computing nodes of the target computing node.

[0075] Specifically, one or more auxiliary task descriptions can be configured in the task description information of the target computing node, and then the subsequent computing nodes of the target computing node can be assigned to each auxiliary task description. After the target computing node completes its execution, the device scheduler processes each auxiliary task description according to the chained scheduling information of the target computing node; after an auxiliary task description is triggered, the device scheduler processes the task description information of the corresponding subsequent computing node according to the chained scheduling information of each auxiliary task description.

[0076] For example, computing node A has four successor computing nodes: computing node B, computing node C, computing node D, and computing node E. A single task description information can be configured with a maximum of two successor indication information. Auxiliary task description information N1 and auxiliary task description information N2 are generated. N1 and N2 are configured in the task description information S1 of computing node A. Task description information S2 of computing node B and task description information S3 of computing node C are configured in N1. Task description information S4 of computing node D and task description information S5 of computing node E are configured in N2. After computing node A completes its execution, the device scheduler processes N1 and N2 according to task description information S1, then schedules computing nodes B and C according to N1, and schedules computing nodes D and E according to N2.

[0077] In this disclosure, the auxiliary task description information serves only as an intermediate node in the scheduling structure, without altering the set of computing nodes in the task graph or the dependencies between them. By expanding the carrying capacity of subsequent indication information through the auxiliary task description information, even when the number of subsequent computing nodes for the target computing node exceeds the configuration limit of a single task description information, the device scheduler can still advance each subsequent computing node without requiring the host to submit each subsequent computing node individually, thus reducing the startup overhead of task graphs with wide fan-out structures.

[0078] According to some embodiments of this disclosure, in order to enable the auxiliary task description information to handle multiple predecessor dependencies, optionally, in response to the target computing node having at least two predecessor computing nodes as its successor computing node, the dependency action information in the chained scheduling information of the auxiliary task description information is configured to perform a decrement operation on the dependency count value of the successor computing node.

[0079] Specifically, the successor computing node of the target computing node may also have other predecessor computing nodes besides the target computing node. For this successor computing node, the dependency count value in its task description information is initialized to the total number of its predecessor computing nodes. The auxiliary task description information points to the task description information of this successor computing node through its own chained scheduling information, and the corresponding dependency action information is configured as a decrement operation.

[0080] After the target compute node completes its execution, the device scheduler first processes the auxiliary task description information, and then uses this information to decrement the dependency count of the subsequent compute node. After the other predecessor compute nodes of this subsequent compute node complete their execution, they also decrement the dependency count. When the dependency count reaches zero, the device scheduler schedules the task description information for that subsequent compute node.

[0081] For example, computing node C has two predecessor computing nodes, computing node A and computing node B. Computing node A corresponds to multiple successor computing nodes, which need to be expanded through auxiliary task description information N1. The dependency count value in the task description information S3 of computing node C is initialized to 2. S3 is configured in the chained scheduling information of N1, and the corresponding dependency action information is configured as a decrement operation. S3 is also configured in the chained scheduling information of the task description information S2 of computing node B, and the corresponding dependency action information is configured as a decrement operation. After computing node A completes execution, the device scheduler decrements the dependency count value in S3 from 2 to 1 through N1. After computing node B completes execution, the device scheduler decrements the dependency count value from 1 to 0 and schedules S3.

[0082] In this disclosure, the auxiliary task description information can not only expand the number of successor computing nodes that the target computing node can trigger, but also perform a decrement operation on successor computing nodes with multiple predecessor computing nodes, so that the wide fan-out structure and multiple predecessor dependencies can be used in combination without the host intervening in the judgment of the completion status of each predecessor computing node. This can expand the scope of application of chain scheduling while ensuring the correctness of the dependency relationship.

[0083] According to some embodiments of this disclosure, in order to enable the task description information to carry the basic information required to start the computing node, optionally, the step 101 above, which generates corresponding task description information for each computing node in the task graph, may specifically include: Write the program entry address, grid dimension, thread block dimension, resource configuration information, and constant parameter area address of the corresponding computing node into the task description information.

[0084] Specifically, the program launch PC indicates the starting address of the program instructions for the corresponding computing node, and the device scheduler determines the instructions to be executed based on the program launch PC.

[0085] The grid dimension (CTA raster width / height / depth) is used to describe the number and arrangement of thread blocks corresponding to the compute node in each dimension, and the thread block dimension (thread block width / height / depth) is used to describe the number and arrangement of threads in each thread block in each dimension.

[0086] Resource configuration information describes the hardware resources required for compute nodes to execute. In one implementation, resource configuration information may include at least one of the following: register count, barrier count, and shared memory size. The device scheduler can allocate execution resources to the corresponding compute nodes based on the resource configuration information.

[0087] The constant bank is a storage area on the device side used to store the kernel parameters of compute nodes. The address in the constant bank indicates the storage location of the corresponding compute node's kernel parameters on the device side. When the device scheduler schedules a compute node, it retrieves the corresponding kernel parameters based on this address.

[0088] During the instantiation or update phase of the task graph, based on the program and startup configuration corresponding to the compute node, the program entry address, grid dimension, thread block dimension, resource configuration information, and the address of the constant parameter area are extracted and written into the task description information corresponding to that compute node. When the device scheduler subsequently schedules this task description information, it can directly use the information in it to start the corresponding compute node.

[0089] For example, the program entry address of compute node A is AddrA, the grid dimension is 4×1×1, the thread block dimension is 128×1×1, and the resource configuration information includes the number of registers required by compute node A and the size of the shared memory space. The address of the constant parameter area is AddrP. This information is written into the task description information S1 corresponding to compute node A. When the device scheduler schedules the task description information S1, it obtains the program instructions of compute node A based on AddrA, organizes the execution structure of compute node A according to the grid dimension and thread block dimension, and allocates execution resources according to the resource configuration information.

[0090] In this disclosure, the task description information carries the program entry point, execution structure, resource requirements, and startup parameter location of the computing node. The device scheduler can start the corresponding computing node based on the task description information, without the host needing to reorganize the above information each time the task graph is started, which can reduce the repetitive processing during the task graph startup phase.

[0091] In one implementation, in order to enable the task description information to carry the basic startup information and chained scheduling information of the computing node in a unified data structure, the task description information generated in step 101 may optionally include: chained scheduling information, program entry address, grid dimension, thread block dimension, resource configuration information, and address of constant parameter area.

[0092] The chained scheduling information includes at least one successor configuration item, a dependency count value, a scheduling trigger flag, and self-recovery indication information.

[0093] The subsequent configuration items include subsequent instruction information, dependency action information, enable flags, and prefetch instruction information, which correspond one-to-one. The prefetch instruction information can include an instruction prefetch field and a constant parameter prefetch field. The instruction prefetch field is used to indicate the instruction corresponding to the subsequent compute node, and the constant parameter prefetch field is used to indicate the constant parameter corresponding to the subsequent compute node.

[0094] According to some embodiments of this disclosure, in order to prevent the pre-loaded task description information from being scheduled prematurely, the chained scheduling information may optionally include a scheduling trigger flag, and step 102 above may further include: Configure a scheduling trigger flag (require Scheduling LCAS) in the task description information. The scheduling trigger flag is used to indicate that the corresponding task description information will not enter the scheduling queue after being loaded to the device side. Before submitting the task description information of the entry computing node of the task graph, the task description information of each computing node other than the entry computing node is preloaded to the device side. For task description information configured with the scheduling trigger flag, the device scheduler performs a scheduling operation on the task description information based on the chain scheduling information of the predecessor computing node corresponding to the task description information, so that the task description information enters the scheduling queue.

[0095] Here, the task description information also includes a loading status and a schedulable status. The loading status indicates that the task description information has been loaded to the device side, and the device scheduler can access the task description information; the schedulable status indicates that the task description information has entered the execution queue, and the device scheduler can start the corresponding compute node.

[0096] For task descriptions configured with a scheduling trigger flag, even if the task description has already been loaded onto the device side, it will not directly enter the scheduling queue. Only when the predecessor compute node corresponding to the task description completes its execution, and the device scheduler performs a scheduling operation on the task description based on the chained scheduling information of the predecessor compute node, will the task description enter the scheduling queue.

[0097] Specifically, when initiating the task graph, before submitting the task description information of the entry computing node, the host first loads the task description information of all computing nodes except the entry computing node onto the device side. Since the aforementioned task description information is configured with a scheduling trigger flag, it does not immediately enter the scheduling queue after loading. The host then submits the task description information of the entry computing node, and the device scheduler starts the entry computing node. After any computing node completes execution, the device scheduler, based on its chained scheduling information, performs scheduling operations on the loaded task description information corresponding to the subsequent computing node, causing that task description information to enter the scheduling queue.

[0098] For example, the task graph includes sequentially dependent computing nodes A, B, and C, with computing node A as the entry computing node. When starting the task graph, the task description information S2 corresponding to computing node B and the task description information S3 corresponding to computing node C are first loaded to the device side. Both task description information S2 and task description information S3 are configured with scheduling trigger flags, so they do not enter the scheduling queue after loading. Then, the task description information S1 corresponding to computing node A is submitted. After computing node A completes execution, the device scheduler performs a scheduling operation on task description information S2 based on the chained scheduling information in task description information S1; after computing node B completes execution, the device scheduler performs a scheduling operation on task description information S3 based on the chained scheduling information in task description information S2.

[0099] In this disclosure, a scheduling trigger flag is used to distinguish between the loading and schedulable states of task description information. This allows the task description information of non-entry computing nodes to be loaded onto the device side in advance, while preventing them from being scheduled prematurely before their dependency conditions are met. This reduces loading wait times during the chained execution process and ensures that each computing node executes according to the dependencies in the task graph.

[0100] According to some embodiments of this disclosure, in order to support repeated startup of the task graph, the chained scheduling information may optionally include self-recovery indication information, and step 102 above may also include: configuring self-recovery indication information (self Copy On Completion) in the task description information of each computing node except the ingress computing node.

[0101] The self-recovery indication information is used to instruct the device scheduler to read the original task description information stored on the device side after the corresponding computing node has finished executing, and restore the task description information of the computing node to a reschedulable state, so that when the task graph is restarted, the task description information that has been restored to a reschedulable state does not need to be reloaded to the device side.

[0102] Here, the original task description information refers to the task description information generated and stored on the device side during the instantiation or update phase of the task graph. During the execution of a compute node, the scheduling status of its corresponding loaded task description information may change; for task description information with dependency count values, its dependency count value may also decrease as the predecessor compute node completes.

[0103] For task descriptions configured with self-recovery indications, after the corresponding compute node completes execution, the device scheduler reads the original task description information stored on the device side and restores the task description information of the compute node based on the original task description information. For example, the scheduling status of the task description information is restored to the state corresponding to the waiting scheduling trigger flag; for task descriptions with dependency count values, their dependency count values ​​are restored to their initial values.

[0104] When the task graph is restarted, the task description information that has been restored to a reschedulable state remains on the device side, without requiring the host to reload. After the host submits the task description information of the entry compute node, the device scheduler can advance the task graph again according to the chained scheduling information between the compute nodes.

[0105] For example, compute node A is the entry compute node, and compute nodes B and C are non-entry compute nodes. Both task description information S2 and task description information S3 are configured with self-recovery indication information. When the task graph is started for the first time, compute nodes B and C complete their execution sequentially. The device scheduler then restores task description information S2 and task description information S3 to a reschedulable state based on the original task description information stored on the device side. When the task graph is started for the second time, the host does not need to reload task description information S2 and task description information S3; it only needs to submit the task description information S1 corresponding to the entry compute node A.

[0106] In this disclosure, the task description information of non-entry computing nodes is automatically restored after the corresponding computing node completes execution. This eliminates the need for the host to reload or reinitialize the task description information each time the task graph is started, reducing loading overhead and host processing overhead when repeatedly starting the task graph. Simultaneously, dependency counts and scheduling states can be restored to their initial states, ensuring that the dependencies between computing nodes are accurately represented each time the task graph is started.

[0107] According to some embodiments of this disclosure, in order to reduce the update overhead when repeatedly starting the task graph, optionally, during the instantiation phase, the task description information of each computing node and the startup parameters stored in the constant parameter area corresponding to each task description information are recorded. The constant parameter area is a storage area on the device side used to store the startup parameters of the computing nodes. Before submitting the task description information of the entry computing node of the task graph, the content that has changed from the previous startup is determined. In response to the change of startup parameters, the changed startup parameters are written into the corresponding constant parameter area. In response to the change of static fields in the task description information, the corresponding task description information is updated. For the content that has not changed, the content recorded during the instantiation phase is reused.

[0108] Specifically, during the task graph instantiation phase, in addition to generating task description information for each compute node, the startup parameters stored in the constant parameter area corresponding to each task description information are also recorded. Startup parameters may include, for example, the storage address that the compute node needs to access, the data length, or other runtime parameters.

[0109] Before restarting the task graph, the content required for this startup is compared with the content used in the previous startup to determine what has changed. If only the startup parameters have changed, a parameter patch is performed on the corresponding constant parameter area, writing the changed startup parameters into the corresponding constant parameter area; no modification to the corresponding task description information is required. If the static fields in the task description information have changed, the corresponding task description information is updated. Static fields may include at least one of the following: program entry address, grid dimension, thread block dimension, resource configuration information, and the address of the constant parameter area.

[0110] For startup parameters and static fields that have not changed, the content already recorded during the instantiation phase can be reused directly without regenerating or rewriting.

[0111] For example, compute node A accessed storage address P1 during the previous startup, and accessed storage address P2 during this startup, while other startup parameters and static fields remained unchanged. Before starting the task graph, P1 stored in the constant parameter area was updated to P2, and the task description information S1 corresponding to compute node A was reused. If the grid dimension of compute node A changed, the grid dimension field in the task description information S1 was updated.

[0112] In this disclosure, task description information and startup parameters are recorded during the instantiation phase. When the task graph is restarted, only the changed content is updated and the unchanged content is reused. There is no need to regenerate all task description information or rewrite all startup parameters, which can reduce the amount of data written and host processing overhead when the task graph is repeatedly started.

[0113] According to some embodiments of this disclosure, in order to ensure that the updated task description information is correctly scheduled, optionally, the task description information of the successor computing node of the computing node in step 103 above may specifically include: When the task description information is updated, the device scheduler performs a scheduling operation on the task description information after performing an invalidation operation and a copy operation on the task description information.

[0114] Specifically, if the task description information of the subsequent computing node is updated before this startup, but the old content of the task description information is still stored in the device-side scheduling cache, the device scheduler first performs an invalidation operation to invalidate the old task description information in the device-side scheduling cache; then it performs a copy operation to load the updated task description information into the device-side scheduling cache; finally, it performs a scheduling operation to put the updated task description information into the scheduling queue.

[0115] The failure operation, copy operation, and scheduling operation are executed in the order described above. The device scheduler will only schedule the task description information after the old task description information has become invalid and the updated task description information has been loaded.

[0116] For example, the static fields in the task description information S2 of compute node B were updated before this startup, and the old content of the task description information S2 is stored in the device-side scheduling cache. After compute node A finishes execution, the device scheduler needs to first invalidate the old content of the task description information S2, then load the updated task description information S2 into the device-side scheduling cache, and finally schedule the updated task description information S2 so that compute node B executes according to the updated configuration.

[0117] In this disclosure, the device scheduler processes the updated task description information in the order of first invalidating, then copying, and then scheduling. This avoids the old task description information in the device-side scheduling cache being incorrectly scheduled, ensuring that the computing node executes according to the updated content corresponding to this startup.

[0118] According to some embodiments of this disclosure, in order to reduce the waiting time when subsequent computing nodes start up, step 102 may optionally include: Configure dependent DESC prefetch information in the task description information of the compute node. The prefetch information is used to instruct the device scheduler to perform instruction prefetching and constant parameter prefetching before scheduling the task description information of the subsequent compute node, and to prefetch the instructions and constant parameters corresponding to the subsequent compute node.

[0119] Specifically, the prefetch instruction information can indicate whether to prefetch the instructions corresponding to the successor computing node, whether to prefetch the constant parameters corresponding to the successor computing node, or uniformly indicate that both instructions and constant parameters should be prefetched.

[0120] After a compute node completes its execution, before scheduling the task description information of its successor compute node, the device scheduler obtains the corresponding instructions and constant parameters of the successor compute node based on the prefetch instruction information in the task description information of the previous compute node, and prefetches them into the corresponding cache on the device side. After completing the prefetch, the device scheduler then schedules the task description information of the successor compute node.

[0121] For example, computing node A is the predecessor computing node of computing node B, and the task description information S1 of computing node A is configured with prefetching indication information. After computing node A completes its execution, the device scheduler prefetches the corresponding instructions and constant parameters of computing node B before scheduling the task description information S2 of computing node B, and then schedules the task description information S2.

[0122] In this disclosure, the device scheduler obtains the instructions and constant parameters required by the successor computing node before scheduling the successor computing node. This reduces cache misses and read waits when the successor computing node starts up, making the execution between adjacent computing nodes more compact and improving the utilization of parallel computing devices.

[0123] According to some embodiments of this disclosure, in order to enable different types of nodes in the task graph to be scheduled through task description information, optionally, the computing nodes in the task graph may include kernel node, memcpy node, and memset node; the step 101 above, which generates corresponding task description information for each computing node in the task graph, may specifically include: For data copy nodes and data fill nodes, the data copy operation corresponding to the data copy node or the data fill operation corresponding to the data fill node is converted into a copy operation or fill operation that can be executed on the device side, and corresponding task description information is generated for the converted operation.

[0124] Specifically, kernel function compute nodes are used to execute corresponding kernel functions. For a kernel function compute node, corresponding task description information can be generated based on its program entry point, execution structure, and resource configuration.

[0125] Data copy nodes are used to copy data from a source storage location to a target storage location. For a data copy node, based on the address of the source storage location, the address of the target storage location, and the amount of data to be copied, the data copy operation corresponding to the data copy node is converted into a copy operation that can be executed on the device side, and corresponding copy task description information (copy DESC) is generated for the converted copy operation.

[0126] The data fill node is used to set the data in the target storage area to a specified value. For a data fill node, based on the address of the target storage area, the fill value, and the range of data to be filled, the data fill operation corresponding to the data fill node is converted into a fill operation that can be executed on the device side, and corresponding fill task description information (fill DESC) is generated for the converted fill operation.

[0127] After generating task description information for the converted copy or fill operation, chain scheduling information can be configured in the task description information based on the dependency relationship between the data copy node or data fill node and other computing nodes, so that it can be chained and scheduled by the device scheduler like other computing nodes.

[0128] For example, the task graph sequentially includes a kernel function computation node K1, a data copy node M1, and a kernel function computation node K2. Task description information S1 is generated for kernel function computation node K1. The data copy operation corresponding to data copy node M1 is converted into a device-side executable copy operation and task description information S2 is generated. Task description information S3 is generated for kernel function computation node K2. Based on the dependencies between K1, M1, and K2, chained scheduling information pointing to task description information S2 is configured in task description information S1, and chained scheduling information pointing to task description information S3 is configured in task description information S2. When the task graph is started, the device scheduler proceeds sequentially according to task description information S1, task description information S2, and task description information S3.

[0129] In this disclosure, kernel function computation nodes, data copy nodes, and data filling nodes can all generate corresponding task description information and configure chained scheduling information, so that computation, data copying, and data filling operations in the task graph can be uniformly promoted by the device scheduler without the host submitting start commands for different types of nodes, thus expanding the applicability of the task graph chained start method.

[0130] According to some embodiments of this disclosure, in order to utilize the device-side chained scheduling capability while being compatible with task graphs that cannot be fully chained, optionally, the instantiation or update phase of the task graph may also include... Figure 2 Step 201 shown above, and step 103 above, may specifically include the following steps 202-203: Step 201: Divide the task graph into at least one task graph chain segment based on the chaining condition; Step 202: The device scheduler schedules the task description information of each computing node in the same task graph segment in sequence according to the chain scheduling information. Step 203: For adjacent first task graph chain segments and second task graph chain segments, after the computation node of the first task graph chain segment has completed execution, the task description information of the entry computation node of the second task graph chain segment is submitted through a synchronization command.

[0131] The chaining conditions may include: the dependencies between computing nodes have been determined in the instantiation or update phase, and the dependencies can be expressed through chained scheduling information. Computing nodes that meet the chaining conditions are assigned to the same task graph segment, while computing nodes that do not meet the chaining conditions are treated as segment boundaries.

[0132] Here, a graph chain segment refers to a subgraph within the task graph that can be continuously advanced by the device scheduler according to chained scheduling information. The dependencies between computational nodes within a graph chain segment should be determined during the instantiation or update phase of the task graph and can be expressed through chained scheduling information in the task description. For nodes whose execution paths require runtime judgment, immediate host processing, or cannot be expressed through chained scheduling information, these nodes can be designated as boundary nodes of the graph chain segment.

[0133] Specifically, in step 101, each computing node can be traversed according to the topological order of the task graph: for the current computing node, if the dependency relationship between the current computing node and its predecessor computing node has been determined and the dependency relationship can be configured as chained scheduling information, then the current computing node and its predecessor computing node are assigned to the same task graph chain segment; if the current computing node does not meet the chaining condition, or the dependency relationship between the current computing node and its predecessor computing node cannot be expressed by chained scheduling information, then the boundary of the task graph chain segment is formed at the current computing node.

[0134] In step 103, the device scheduler can sequentially advance each computing node within the same task graph segment according to the chained scheduling information in the task description information. Synchronization commands may include, for example, launchbuffer commands, event commands, or semaphore commands.

[0135] For example, the task graph sequentially includes compute node A, compute node B, boundary node X, compute node C, and compute node D. Compute nodes A and B satisfy the chaining condition and are assigned to the first task graph segment; compute nodes C and D satisfy the chaining condition and are assigned to the second task graph segment; boundary node X does not satisfy the chaining condition and serves as the boundary node between the two task graph segments. When the task graph is started, the device scheduler sequentially schedules compute nodes A and B within the first task graph segment. After the synchronization processing corresponding to boundary node X is completed, the task description information of compute node C is submitted via a synchronization command, and then the device scheduler continues to schedule compute node D within the second task graph segment.

[0136] In this disclosure, computing nodes that meet the chaining conditions are sequentially scheduled by the device scheduler according to chaining scheduling information within the same task graph segment. Nodes that do not meet the chaining conditions serve as boundary nodes and are connected to adjacent task graph segments via synchronization commands. Therefore, it is not necessary for the entire task graph to be chainable; only the chainable portions of the task graph can be scheduled on the device side, ensuring the startup efficiency of the task graph and compatibility with complex topologies.

[0137] According to some embodiments of this disclosure, in order to divide the task graph into at least one task graph segment based on the hardware capabilities of the target device, step 201 above, which divides the task graph into at least one task graph segment based on chaining conditions, may optionally include... Figure 3 Steps 301-303 are shown below: Step 301: Obtain the chain scheduling capability parameters of the target device. The chain scheduling capability parameters include the upper limit of the number of successor indication information that can be configured for a single task description information, and the upper limit of the number of predecessors supported by the device scheduler for multiple predecessor dependencies (max DESC Join Degree). Step 302: In response to the number of successor computing nodes or predecessor computing nodes of a computing node exceeding the corresponding upper limit in the chain scheduling capability parameter, disconnect the task graph chain segment at the computing node. Step 303: Insert an auxiliary computing task (helperkernel) between the computing node and its successor computing node, so that after the computing node finishes execution, the scheduling of the task description information of the successor computing node is triggered based on the auxiliary computing task; or, after the computing node finishes execution, the host submits the task description information of the successor computing node.

[0138] Specifically, during the instantiation or update phase of the task graph, the chained scheduling capability parameters of the target device can be obtained. These chained scheduling capability parameters are hardware capability parameters related to the device scheduler in the target device, which are used to characterize the device scheduler's ability to perform scheduling based on chained scheduling information.

[0139] For any given compute node, count the number of its successor compute nodes and the number of its predecessor compute nodes. If the number of successor compute nodes exceeds the maximum number of successor indications that can be configured in a single task description, it indicates that all successor relationships cannot be directly expressed through the task description information of that compute node. If the number of predecessor compute nodes exceeds the maximum number of predecessors supported by the device scheduler for multiple predecessor dependencies, it indicates that all predecessor relationships cannot be directly expressed through the dependency count.

[0140] If any of the above quantities exceed the corresponding upper limit, the task graph segment is disconnected at that computing node. For a disconnected successor computing node, after the computing node completes its execution, an auxiliary computing task is executed first, and then the auxiliary computing task triggers the scheduling of task description information for one or more successor computing nodes. The auxiliary computing task can submit the corresponding task description information of the successor computing node to the device scheduler based on a pre-configured set of successor computing nodes.

[0141] In another implementation, auxiliary computing tasks may not be inserted. Instead, after the computing node has completed its execution, the host submits the task description information of the subsequent computing node according to the dependencies in the task graph.

[0142] For example, computing node A has multiple successor computing nodes, and the number of successor computing nodes exceeds the chained scheduling range supported by the target device. The current task graph chain segment is disconnected at computing node A, and auxiliary computing tasks H are inserted between computing node A and each successor computing node. After computing node A completes its execution, auxiliary computing task H executes, triggering the scheduling of task description information for each successor computing node.

[0143] Different target devices can have different chain scheduling capability parameters. When the same task graph is instantiated on different target devices, different task graph segmentation results can be obtained based on the chain scheduling capability parameters of the corresponding target devices.

[0144] In this disclosure, the disconnection position of the task graph chain segment is determined based on the chain scheduling capability parameters of the target device, and auxiliary computing tasks are inserted or subsequent computing nodes are submitted by the host at the position where chain scheduling can no longer continue. This can avoid the dependencies in the task graph from exceeding the hardware expression capabilities of the device and ensure the correctness and compatibility of the task graph execution on different devices.

[0145] According to some embodiments of this disclosure, to avoid excessive cache pressure on the device side due to excessively long task graph segments, step 201 above, which divides the task graph into at least one task graph segment based on chaining conditions, may optionally include: In response to the number of consecutive computing nodes within the same task graph segment reaching the number threshold corresponding to the description information cache granularity of the device scheduler, or if the computing node is configured with segmentation indication information, the current task graph segment is disconnected at the computing node, and the computing node is used as the entry computing node for the new task graph segment.

[0146] Here, the description information cache granularity is used to characterize: the range of task description information that the device scheduler in the target device can effectively maintain during a chained scheduling process; the quantity threshold corresponding to the description information cache granularity can be determined based on the description information cache capacity, cache management granularity, or scheduling strategy of the target device.

[0147] In step 201, a group break counter can be maintained to count the number of consecutive compute nodes within the current task graph segment. When adding compute nodes to the current task graph segment according to the topological order of the task graph, if the current compute node causes the number of consecutive compute nodes within the task graph segment to reach a threshold, the current task graph segment is broken at the current compute node, and the current compute node is used as the entry compute node for the new task graph segment.

[0148] In another implementation, compute nodes can be configured with force group split information, which explicitly indicates that the current task graph segment should be disconnected at that compute node. When a compute node with the force group split information is encountered, the current task graph segment should be disconnected at that compute node, regardless of whether the number of compute nodes in the current task graph segment has reached a certain threshold, and that compute node should be used as the entry compute node for the new task graph segment.

[0149] For example, the threshold number corresponding to the granularity of the target device's description information cache is M. The current task graph segment already includes M-1 consecutive computing nodes, and the currently processed computing node is the Mth computing node. The current task graph segment is disconnected at the Mth computing node, and the Mth computing node is used as the entry computing node for the new task graph segment.

[0150] In this disclosure, actively disconnecting task graph segments based on the quantity threshold or segmentation indication information corresponding to the granularity of the description information cache can avoid the need for the device side to maintain too much task description information at the same time due to the excessive length of a single task graph segment, thereby reducing the cache pressure on the device side and ensuring that the chained scheduling state of the task description information can be correctly maintained.

[0151] According to some embodiments of this disclosure, in order to avoid incorrect chaining of dependencies across scheduling groups, the chaining conditions may optionally include: the computing node and its predecessor computing node belong to the same scheduling group; in response to the computing node and its predecessor computing node belonging to different scheduling groups, the computing node is determined as the entry computing node of a new task graph chain segment.

[0152] Here, a scheduling group is a grouping used by the device scheduler to manage and schedule compute nodes. Compute nodes belonging to the same scheduling group can be continuously scheduled by the device scheduler according to the chained scheduling information in the same scheduling context; compute nodes belonging to different scheduling groups may need to be connected through synchronization commands or host processing at the scheduling group boundary.

[0153] In step 201, when determining whether the current compute node can join the task graph segment where its predecessor compute node is located, the scheduling group to which the current compute node belongs and the scheduling group to which its predecessor compute node belongs are obtained. If they belong to the same scheduling group and other chaining conditions are also met, the current compute node is joined to the task graph segment where the predecessor compute node is located; if they belong to different scheduling groups, the current compute node is not joined to the task graph segment where the predecessor compute node is located, but instead, the current compute node is determined as the entry compute node of the new task graph segment.

[0154] For example, compute node A and compute node B belong to scheduling group G1, and compute node C belongs to scheduling group G2. Compute node B is the predecessor compute node of compute node C. Since compute node C and compute node B belong to different scheduling groups, compute node A and compute node B are assigned to the same task graph segment, and compute node C is determined as the entry compute node of the new task graph segment.

[0155] In this disclosure, computing nodes belonging to the same scheduling group are assigned to the same task graph segment. When a computing node and its predecessor computing node belong to different scheduling groups, a new task graph segment is formed. This can prevent the device scheduler from directly advancing computing nodes across scheduling groups and ensure the correct scheduling order and synchronization semantics between different scheduling groups.

[0156] According to some embodiments of this disclosure, in order to process task graphs with runtime branches, optionally, the boundary nodes may include condition control nodes, the execution path of which is determined based on the runtime state after the task graph is started; in step 201, dividing the task graph into at least one task graph segment based on chaining conditions may include... Figure 4 The following steps 401-403 are shown: Step 401: According to the chaining conditions, the computational nodes contained in the conditional body of the condition control node are divided, and the computational nodes that meet the chaining conditions are separately divided into task graph chain segments. Step 402: When the condition control node determines the execution condition execution body, the task description information of the entry calculation node of the task graph chain segment corresponding to the condition execution body is submitted using the auxiliary calculation task. Step 403: After the task graph segment corresponding to the condition executor has been executed, submit the task description information of the subsequent computing nodes through the synchronization command.

[0157] Here, the conditional node is used to determine whether to execute the corresponding conditional execution body, or which execution path within the conditional execution body, based on the runtime state after the task graph starts. Since the execution path of the conditional node cannot be determined during the task graph instantiation or update phase, the conditional node is not treated as a regular computation node within the task graph chain, but rather as a boundary node of the task graph chain.

[0158] The condition execution body may include one or more computation nodes. In step 201, the computation nodes within the condition execution body may be divided according to chained conditions, and the computation nodes that satisfy the chained conditions may be separately divided into task graph segments. The entry and exit points of the condition control nodes are not directly incorporated into the task graph segments within the condition execution body.

[0159] When the condition control node determines the condition executor to execute based on the runtime state, the auxiliary computing task submits the task description information of the entry computing node of the task graph segment corresponding to the condition executor. The device scheduler then schedules each computing node within that task graph segment sequentially based on the chained scheduling information within that segment. After the task graph segment is completed, it submits the task description information of the subsequent computing nodes following the condition executor via a synchronization command.

[0160] For example, the task graph includes a condition control node X, and the condition execution body of condition control node X includes sequentially dependent computing nodes B and C, followed by computing node D. Computing nodes B and C are separately divided into a task graph chain segment. When condition control node X determines to execute the condition execution body, the auxiliary computing task submits the task description information of computing node B; after computing node B completes execution, the device scheduler schedules computing node C according to the chained scheduling information; after computing node C completes execution, it submits the task description information of computing node D through a synchronization command.

[0161] In this disclosure, the condition control node is used as the boundary node, and the computing nodes that meet the chaining conditions within the condition execution body are separately divided into task graph segments. This allows the static dependencies within the condition execution body to still be promoted in a chain by the device scheduler, while the runtime judgment of the condition control node is handled by auxiliary computing tasks and synchronization commands, thus balancing the correctness of the dynamic control flow and the startup efficiency of the task graph within the condition execution body.

[0162] According to some embodiments of this disclosure, in order to maintain consistency between task description information and chain dependencies after the task graph is updated, optionally, in step 101 above, generating corresponding task description information for each computing node in the task graph during the task graph update phase may include: In response to updates to the startup parameters or resource configuration of a compute node that do not match the task description information already generated by the compute node, the task description information of the compute node and its direct successor compute nodes is regenerated, and the corresponding chain scheduling information is reconfigured. If the task graph has been divided into task graph segments, in response to the change in the chain dependencies within the task graph segments caused by the reconfigured chain scheduling information, the task description information of each computing node located after the computing node in the task graph segment is regenerated.

[0163] Specifically, during the task graph update phase, the updated startup parameters or resource configuration of the compute node can be obtained, and this updated result can be matched with the task description information already generated for that compute node. If the update result can be achieved by repairing the constant parameter area, and the already generated task description information can still correctly represent the compute node, then the already generated task description information can be reused; if the update result does not match the already generated task description information, then the task description information for that compute node needs to be regenerated.

[0164] When the task description information of a compute node changes, the chain dependency relationship of its direct successor compute nodes may also change. Therefore, when regenerating the task description information of the compute node, the task description information of its direct successor compute nodes should also be regenerated, and the chain scheduling information between the compute node and its direct successor compute nodes should be reconfigured.

[0165] If the task graph has been divided into task graph segments, and the reconfigured chain scheduling information causes changes in the chain dependencies within the task graph segments, then the task description information of each computing node following that computing node in the task graph segment is regenerated to ensure that the chain dependencies following that computing node remain consistent.

[0166] For example, a task graph segment includes sequentially dependent compute nodes A, B, and C. The resource configuration of compute node A is updated, and the updated result does not match the task description information S1 already generated by compute node A. The task description information S1 of compute node A and its direct successor compute node B's task description information S2 are regenerated, and the chained scheduling information between them is reconfigured. If the reconfigured chained scheduling information further causes a change in the chained dependency relationship between compute node B and compute node C, then the task description information S3 of compute node C is regenerated.

[0167] In this disclosure, when the task graph update causes a mismatch in the generated task description information, only the task description information of the affected computing nodes and their direct successor computing nodes is regenerated. When the chain dependency relationship within the task graph chain segment changes, the task description information of the subsequent computing nodes is updated. It is not necessary to regenerate all the task description information of the entire task graph, which can reduce the task graph update overhead and ensure that the updated chain dependency relationship is correct.

[0168] According to some embodiments of this disclosure, in order to support the statistics and replay of the task graph execution process, optionally, before starting the task graph, the following may also be included: Insert instrumentation-assisted task description information into the chained dependency relationship formed by the task description information of each computing node, and configure chained scheduling information for the instrumentation-assisted task description information.

[0169] Among them, the instrumentation helper DESC is used to collect statistics on the execution information of computing nodes or control the replay of the task graph; after inserting the instrumentation helper DESC, the set of computing nodes in the task graph remains unchanged.

[0170] Here, the instrumentation-assisted task description information does not correspond to any existing computation node in the task graph. Instead, it is used in conjunction with a performance profiler or replay tool to statistically analyze or control the execution process of the task graph. In one implementation, the instrumentation-assisted task description information may include counter update task description information (DESC), used to statistically analyze the start execution information, completion execution information, or execution count of computation nodes; it may also include replay control task description information (DESC), used to control the replay process of the task graph.

[0171] Specifically, instrumentation auxiliary task description information can be inserted between the predecessor computing node and the successor computing node. During insertion, the chained scheduling information in the predecessor computing node's task description information is adjusted to point to the instrumentation auxiliary task description information, and the instrumentation auxiliary task description information is configured with chained scheduling information pointing to the original successor computing node's task description information. Therefore, after the predecessor computing node completes its execution, the device scheduler first processes the instrumentation auxiliary task description information before continuing to schedule the original successor computing node.

[0172] For example, the task graph includes sequentially dependent computing nodes A and B. Before starting the task graph, instrumentation-assisted task description information I is inserted between computing nodes A and B. The chained scheduling information in the task description information S1 of computing node A is adjusted to point to instrumentation-assisted task description information I, and chained scheduling information pointing to task description information S2 of computing node B is configured in instrumentation-assisted task description information I. After computing node A completes execution, the device scheduler first processes instrumentation-assisted task description information I to collect execution information of computing node A, and then schedules task description information S2 according to instrumentation-assisted task description information I.

[0173] In this disclosure, instrumentation assists in inserting task description information into the chained dependencies formed by task description information. The original set of computation nodes in the task graph remains unchanged, as do the user-visible dependencies between computation nodes. Therefore, execution information of computation nodes can be statistically analyzed or task graph replay can be controlled without altering the original semantics of the task graph, reducing the additional startup overhead during performance analysis and replay.

[0174] According to some embodiments of this disclosure, in order for the device scheduler to directly access the task description information of each computing node, step 101, generating corresponding task description information for each computing node in the task graph, may optionally include: The generated task description information is stored in a description information storage pool accessible on the device side, and the storage address indicated by the subsequent instruction information is the address in the description information storage pool.

[0175] Here, the description information storage pool (DESC pool) is a device-accessible storage area used to centrally store task description information corresponding to each compute node in the task graph. The description information storage pool can be allocated during the instantiation or update phase of the task graph, and a corresponding storage address can be assigned to each task description information.

[0176] After generating the task description information in step 101, the task description information is written to the description information storage pool. In step 102, when configuring the chained scheduling information, the storage address of the task description information of the successor computing node in the description information storage pool is written to the successor indication information of the predecessor computing node. The device scheduler can then access the corresponding task description information in the description information storage pool based on the successor indication information.

[0177] For example, task description information S1 of computing node A is stored at address P1 in the description information storage pool, and task description information S2 of computing node B is stored at address P2 in the description information storage pool. If computing node A is the predecessor computing node of computing node B, then address P2 is written into the successor indication information of task description information S1. After computing node A completes execution, the device scheduler retrieves task description information S2 from the description information storage pool based on address P2.

[0178] In this disclosure, the task description information is centrally stored in a description information storage pool accessible on the device side. The subsequent indication information directly indicates the address in the description information storage pool, enabling the device scheduler to directly access the task description information of the subsequent computing node according to the chained scheduling information. This eliminates the need for the host to transmit task description information one by one during the task graph startup phase, thereby reducing the task graph startup overhead.

[0179] According to some embodiments of this disclosure, in order to handle errors during the execution of computing nodes, the above method may optionally further include: In response to an error in the execution of a compute node, the device scheduler stops scheduling compute nodes that belong to the same context as the compute node that malfunctioned. In response to the context being reset, the device scheduler resumes scheduling of the compute nodes within the context.

[0180] Here, context represents a set of scheduling states and execution resources associated with the execution of a task graph. There may be chained dependencies between compute nodes belonging to the same context. If one compute node encounters an error, continuing to schedule subsequent compute nodes within that context may amplify the error.

[0181] Specifically, errors in compute node execution can include, for example, illegal memory access. When the device scheduler detects an error, it records the error status and stops scheduling compute nodes belonging to the same context as the affected node, refraining from issuing new schedules to compute nodes in that context that have not yet been executed.

[0182] Users or debugging tools can read the error status of this context. After the context is reset, the error status in the context is cleared, and the device scheduler resumes scheduling of the compute nodes within the context, enabling the task graph to restart or continue execution.

[0183] For example, compute node B experiences an illegal memory access during execution. The device scheduler stops scheduling compute nodes C and D, which belong to the same context as compute node B. After the user reads the error status of compute node B, the context is reset, and the device scheduler resumes scheduling compute nodes B, C, and D.

[0184] In this disclosure, when a compute node encounters an error, only the compute nodes belonging to the same context as the erroneous compute node are stopped from scheduling, and scheduling is resumed after the context is reset. This can prevent the erroneous compute node from causing subsequent compute nodes to continue executing, and assists in the error location and rapid recovery of the task graph.

[0185] According to some embodiments of this disclosure, in order to support multiple executions of the conditional executor, optionally, after each computing node in the task graph chain segment corresponding to the conditional executor has completed execution, the device scheduler reads the original task description information stored on the device side and restores the task description information of the computing node to a state that can be rescheduled, so as to support multiple executions of the conditional executor.

[0186] Here, a condition control node can determine to execute the same conditional executor multiple times during a single execution of the task graph. After each execution of the task graph segment corresponding to the conditional executor, the scheduling state of its task description information may change; for task description information with dependency counts, the dependency count may have already decreased to zero. To enable the task graph segment to be executed again, its task description information needs to be restored to a reschedulable state.

[0187] Specifically, after the computation node in the task graph chain segment corresponding to the conditional executor completes its execution, the device scheduler reads the original task description information stored on the device side and restores the task description information of the corresponding computation node based on the original task description information. For task description information with dependency count values, the dependency count values ​​can be restored to their initial values; for task description information with scheduling trigger flags, their scheduling status can be restored to the state of waiting for scheduling trigger.

[0188] In one implementation, after any computing node within the task graph segment corresponding to the conditional executor completes execution, the device scheduler can read and restore the original task description information of that computing node. When all computing nodes within the task graph segment have completed execution, the task description information of each task in the task graph segment is restored to a reschedulable state. When the conditional control node determines to execute the conditional executor again, the auxiliary computing task can resubmit the task description information of the entry computing node of the task graph segment.

[0189] For example, the task graph segment corresponding to the conditional execution body includes sequentially dependent computing nodes B and C. When the conditional execution body is executed for the first time, computing nodes B and C complete their execution sequentially. The device scheduler, based on the original task description information stored on the device side, restores task description information S2 and task description information S3 to a reschedulable state. When the conditional control node determines to execute the conditional execution body for the second time, the auxiliary computing task resubmits task description information S2, and the device scheduler then schedules task description information S3 according to the chained scheduling information.

[0190] In this disclosure, after the task graph chain segment corresponding to the conditional executor is executed, the device scheduler restores the task description information of each computing node in the chain segment according to the original task description information, so that the conditional executor can be executed multiple times without the host reloading or re-initializing the task description information before each execution. This reduces the startup overhead when the conditional executor is repeatedly executed and ensures that the chain dependency relationship is accurately expressed in each execution.

[0191] According to some embodiments of this disclosure, if the target device does not support scheduling task description information according to chained scheduling information, then when starting the task graph, the host submits the task description information of each computing node one by one according to the dependencies between nodes.

[0192] The following is a detailed explanation of the task graph startup process disclosed herein, using a specific example: The task graph includes compute nodes A, B, C, D, and E. Compute node A is the entry compute node; after compute node A is executed, compute nodes B and C can be executed respectively; compute node D has two predecessor compute nodes, compute nodes B and C; after compute node D is executed, compute node E is executed.

[0193] like Figure 5 As shown, in step 101, during the instantiation or update phase of the task graph, task description information S1 to task description information S5 are generated for computing nodes A to E respectively, and the task description information S1 to task description information S5 are stored in a description information storage pool accessible on the device side.

[0194] In step 102, chained scheduling information is configured based on the dependencies between computing nodes.

[0195] Specifically, the task description information S1 is configured with subsequent indication information pointing to the task description information S2 and the task description information S3 respectively; the task description information S2 is configured with subsequent indication information pointing to the task description information S4; the task description information S3 is configured with subsequent indication information pointing to the task description information S4; and the task description information S4 is configured with subsequent indication information pointing to the task description information S5.

[0196] For the subsequent indication information of task description information S1, if task description information S2 and task description information S3 have been loaded to the device side, the corresponding dependent action information can be configured as a scheduling operation; if task description information S2 and task description information S3 have not yet been loaded to the device side, the corresponding dependent action information can be configured as a copy and scheduling operation.

[0197] For the successor indication information of task description information S2 and task description information S3, the corresponding dependency action information is configured as a decrementing operation, and the dependency count value in task description information S4 is initialized to 2. The dependency count value in task description information S4 is used to record the number of predecessor computing nodes of computing node D that have not yet been completed.

[0198] In step 103, when starting the task graph, the host only submits the task description information S1 corresponding to the entry computing node A. The device scheduler schedules the task description information S1, causing computing node A to execute it.

[0199] After compute node A completes its execution, the device scheduler, based on the chained scheduling information in task description information S1, schedules task description information S2 and task description information S3, causing compute nodes B and C to execute respectively. This process corresponds to... Figure 5 The two branches formed after computing node A.

[0200] After compute node B completes its execution, the device scheduler accesses task description information S4 based on the chained scheduling information in task description information S2, and decrements the dependency count value in task description information S4 from 2 to 1. Since the dependency count value is not 0, it indicates that compute node C has not yet completed its execution, therefore the device scheduler does not schedule task description information S4 for the time being.

[0201] After compute node C completes its execution, the device scheduler accesses task description information S4 based on the chained scheduling information in task description information S3, and decrements the dependency count value in task description information S4 from 1 to 0. Once the device scheduler detects that the dependency count value is 0, it determines that both compute nodes B and C have completed their execution, and then schedules task description information S4 to allow compute node D to execute.

[0202] After compute node D completes its execution, the device scheduler schedules task description information S5 according to the chained scheduling information in task description information S4, causing compute node E to execute. After compute node E completes its execution, the task graph execution is complete.

[0203] In the example above, during the task graph startup phase, the host only submits the task description information S1 corresponding to the entry compute node A. Subsequent compute nodes B through E are all advanced by the device scheduler according to the chained scheduling information. For the branch structure after compute node A, the device scheduler schedules compute nodes B and C respectively according to the task description information S1. For the multi-predecessor merging structure at compute node D, the device scheduler determines that compute node D will be scheduled only after compute nodes B and C have completed execution, based on the dependency count value. For the linear dependency relationship between compute node D and compute node E, the device scheduler directly advances compute node E according to the task description information S4.

[0204] This ensures the correct execution order under branching and merging relationships, while also reducing the overhead of the host submitting task description information node by node.

[0205] If the number of chainable compute nodes in the task graph is N, the number of entry compute nodes or entry task graph segments is R, and the amount of dynamic content to be updated during this startup is P, then in the traditional task graph startup path, the host needs to traverse each compute node and organize the startup commands for each compute node separately, resulting in a host-side processing overhead of approximately O(N). Using the task graph startup method disclosed herein, task description information and chained scheduling information are pre-generated during the instantiation or update phase. During the startup phase, the host mainly submits the task description information of the entry compute node and processes the content that has changed during this startup, reducing the host-side processing overhead to O(R+P).

[0206] The following example will illustrate in detail the effect of the task graph initiation method disclosed herein: Assuming the task graph includes 6 real kernel function computation nodes, such as Figure 6 As shown, the nodes are initNode, branch0, branch1, branch2, branch3, and joinNode, respectively. The entry computing node has 4 successor computing nodes, and the joining computing node has 4 predecessor computing nodes. There are a total of 8 dependency edges.

[0207] If the traditional startup command, description information copy command, scheduling command, and synchronization auxiliary command corresponding to a single computing node occupy a total of about 600 bytes, then the amount of startup buffer data that the traditional startup path needs to write to the task graph each time is about 6 × 600B = 3600B, or about 3.5KB.

[0208] Considering additional control commands such as branching, merging, and synchronization, the actual startup buffer data size can be estimated at approximately 3.5KB to 4.5KB.

[0209] After adopting the scheme disclosed herein, the task description information corresponding to the six real computing nodes is pre-generated during the instantiation or update phase. For a 4-way branch structure, if the number of configurable successor indications for a single task description is insufficient, auxiliary task description information can be generated to hierarchically carry the successor relationship; for a 4-way convergence structure, the dependency count value of the convergence computing node can be initialized to 4, and decremented by each predecessor computing node after execution.

[0210] In a conservative implementation, each task graph startup still requires a small number of loading or scheduling control operations on non-entry task description information. If the trigger command for the entry task description information is approximately 64 bytes, and the compact control command corresponding to each non-entry task description information is approximately 32 bytes, then for an example containing 6 real compute nodes and 2 auxiliary task description information, the data volume for each task graph startup is approximately: 64B + 7 × 32B + 32B to 64B ≈ 320B, which is approximately 0.3KB.

[0211] As a result, the amount of startup buffer data in the traditional startup path is reduced from about 3.5KB to 4.5KB to about 0.3KB to 0.4KB, a reduction of more than 90%.

[0212] For task graphs that require repeated startup, if the task description information of non-entry compute nodes is configured with self-recovery indication information, then after the corresponding compute node completes execution, the device scheduler can restore the task description information to a reschedulable state based on the original task description information stored on the device side. When restarting the task graph, the host can submit only the task description information of the entry compute node. At this time, the data volume of the steady-state startup path is approximately 64B to 128B, or approximately 0.06KB to 0.13KB, which is a reduction of approximately 97% compared to the traditional startup path.

[0213] In terms of startup time, if the host-side processing cost of each compute node in the traditional startup path is approximately 0.8 to 1.5 microseconds, then the basic processing time for 6 compute nodes is approximately 4.8 to 9.0 microseconds. Further considering branching, merging, synchronization commands, and queue operations, the host-side startup time of the traditional startup path can be estimated at approximately 6 to 12 microseconds.

[0214] With the proposed solution, the host machine no longer advances the six compute nodes one by one, but instead primarily performs a small number of parameter updates and submits entry task description information. In a conservative implementation, the host machine startup time can be estimated at approximately 1.5 microseconds to 3 microseconds; in a steady-state scenario where the task description information has been maintained in a reschedulable state through a self-recovery mechanism, the host machine startup time can be further reduced to approximately 0.8 microseconds to 1.5 microseconds. Correspondingly, the host machine startup time of the task graph can be reduced by approximately 70% to 90%.

[0215] Besides reducing the amount of startup buffer data and host-side startup time, this disclosed solution also reduces the occupation of the device task queue. In traditional startup paths, a large number of compute nodes sequentially fill the device task queue. When the queue is full, the host needs to wait for the queue to be emptied. With this disclosed solution, the host mainly submits the task description information of the entry compute node, and subsequent compute nodes are advanced by the device scheduler according to the chained scheduling information. This reduces the occupation of the device task queue by startup commands and lowers the probability of the host waiting for the queue to be emptied.

[0216] It should be noted that the above benefit estimation refers to the startup path of the task graph, not the end-to-end execution time of the complete application. For scenarios with a large number of compute nodes, short execution time of individual compute nodes, and high number of repeated startups of the task graph, the startup path accounts for a higher proportion of the total time, and the benefits of this disclosed solution are more significant. For scenarios with longer execution times of individual compute nodes, the startup time reduction still exists, but its proportion in the end-to-end execution time will decrease accordingly. If the task graph contains many nodes that do not meet the chaining conditions, or if this startup requires updating a lot of task description information, the actual benefit will decrease accordingly, but the host processing overhead can still be reduced through task graph segmentation, incremental updates, and device-side chaining.

[0217] Corresponding to the task graph initiation method provided in this disclosure, this disclosure also provides a task graph initiation device. Specifically, in conjunction with... Figure 7 As shown, the device includes a generation module 701, a configuration module 702, and a submission module 703.

[0218] The generation module 701 is used to generate corresponding task description information for each computing node in the task graph during the instantiation or update phase of the task graph; the configuration module 702 is used to configure chained scheduling information in the task description information according to the dependency relationship between each computing node, wherein the chained scheduling information is used to at least indicate the task description information of the successor computing node of the computing node; and the submission module 703 is used to submit the task description information of the entry computing node of the task graph when the task graph is started, so that the device scheduler can schedule the task description information of the successor computing node of the computing node according to the chained scheduling information in the task description information of the computing node after the computing node has finished executing.

[0219] According to some embodiments of this disclosure, the chained scheduling information includes at least successor indication information and dependent action information. The successor indication information is used to indicate the storage address of the task description information of the successor computing node on the device side, and the dependent action information is used to indicate the hardware action that the device scheduler will perform on the task description information of the successor computing node after the computing node completes.

[0220] According to some embodiments of this disclosure, the configuration module 702 is further configured to: configure an enable flag in the task description information so that after the computing node completes execution, the device scheduler accesses the subsequent indication information where the enable flag is valid and skips the subsequent indication information where the enable flag is invalid. The enable flag and the subsequent indication information correspond one-to-one, and are used to indicate whether the corresponding subsequent indication information is valid.

[0221] According to some embodiments of this disclosure, the hardware action consists of at least one of the following operation types: a scheduling operation for scheduling the task description information of a successor computing node; a copy operation for loading the task description information of a successor computing node into a device-side scheduling cache; an invalidation operation for invalidating the old task description information already loaded in the device-side scheduling cache; and a decrement operation for decrementing the dependency count value in the task description information of the successor computing node by one, wherein the dependency count value is used to record the number of tasks that have not been completed in the predecessor computing nodes of the corresponding computing node.

[0222] According to some embodiments of this disclosure, the chained scheduling information also includes a dependency count value, and the configuration module 702 is further configured to: initialize the dependency count value in the task description information of the target computing node to the number of predecessor computing nodes of the target computing node for a target computing node having multiple predecessor computing nodes; configure successor indication information pointing to the task description information of the target computing node in the task description information of each predecessor computing node; configure the dependency action information in the task description information of each predecessor computing node to perform a decrement operation on the dependency count value of the target computing node, so that the device scheduler schedules the task description information of the target computing node when the dependency count value of the target computing node is decremented to zero.

[0223] According to some embodiments of this disclosure, the configuration module 702 is further configured to: generate at least one auxiliary task description information in response to the number of successor computing nodes of the target computing node exceeding the upper limit of the number of successor indication information configurable by a single task description information, wherein the auxiliary task description information does not correspond to any computing node in the task graph; configure at least one auxiliary task description information in the chained scheduling information of the task description information of the target computing node; configure the task description information of at least some of the successor computing nodes of the target computing node in the chained scheduling information of each auxiliary task description information, so that the device scheduler schedules each successor computing node of the target computing node according to the chained scheduling information of the auxiliary task description information.

[0224] According to some embodiments of this disclosure, the configuration module 702 is further configured to: in response to the fact that the successor computing node of the target computing node has at least two predecessor computing nodes, configure the dependency action information in the chain scheduling information of the auxiliary task description information to perform a decrement operation on the dependency count value of the successor computing node.

[0225] According to some embodiments of this disclosure, the generation module 701 is further configured to: write the program entry address, grid dimension, thread block dimension, resource configuration information and constant parameter area address of the corresponding computing node into the task description information.

[0226] According to some embodiments of this disclosure, the chained scheduling information further includes a scheduling trigger flag, and the configuration module 702 is further configured to: configure the scheduling trigger flag in the task description information, wherein the scheduling trigger flag is used to indicate that the corresponding task description information will not enter the scheduling queue after being loaded to the device side; before submitting the task description information of the entry computing node of the task graph, the configuration module 702 further includes: preloading the task description information of each computing node other than the entry computing node to the device side; for task description information configured with a scheduling trigger flag, the device scheduler performs a scheduling operation on the task description information based on the chained scheduling information of the predecessor computing node corresponding to the task description information, so that the task description information enters the scheduling queue.

[0227] According to some embodiments of this disclosure, the chained scheduling information also includes self-recovery indication information. The configuration module 702 is further configured to: configure self-recovery indication information in the task description information of each computing node except the entry computing node; according to some embodiments of this disclosure, the self-recovery indication information is used to instruct the device scheduler to read the original task description information stored on the device side after the corresponding computing node has finished executing, and restore the task description information of the computing node to a reschedulable state, so that when the task graph is restarted, the task description information that has been restored to a reschedulable state does not need to be reloaded to the device side.

[0228] According to some embodiments of this disclosure, the system further includes: a recording module, used to record, during the instantiation phase, the task description information of each computing node and the startup parameters stored in the constant parameter area corresponding to each task description information, wherein the constant parameter area is a storage area on the device side used to store the startup parameters of the computing nodes; a determining module, used to determine the content that has changed relative to the previous startup before submitting the task description information of the entry computing node of the task graph; a writing module, used to write the changed startup parameters into the corresponding constant parameter area in response to changes in the startup parameters; an updating module, used to update the corresponding task description information in response to changes in the static fields in the task description information; and a reuse module, used to reuse the content recorded during the instantiation phase for content that has not changed.

[0229] According to some embodiments of this disclosure, the task graph initiation device is applied to a computing system, the computing system including a device scheduler, the device scheduler being configured to: perform a scheduling operation on the task description information after performing an invalidation operation and a copy operation on the task description information when the task description information is updated.

[0230] According to some embodiments of this disclosure, the chained scheduling information also includes prefetching indication information, and the configuration module 702 is further configured to: configure prefetching indication information in the task description information of the computing node, wherein the prefetching indication information is used to instruct the device scheduler to prefetch the instructions and constant parameters corresponding to the successor computing node before scheduling the task description information of the successor computing node.

[0231] According to some embodiments of this disclosure, the computation nodes in the task graph include kernel function computation nodes, data copy nodes, and data filling nodes. The generation module 701 is specifically used to: for data copy nodes and data filling nodes, convert the data copy operation corresponding to the data copy node or the data filling operation corresponding to the data filling node into a copy operation or filling operation that can be executed on the device side, and generate corresponding task description information for the converted operation.

[0232] According to some embodiments of this disclosure, during the instantiation or update phase, the system further includes: a partitioning module, configured to partition the task graph into at least one task graph segment based on chaining conditions; computing nodes that satisfy the chaining conditions are assigned to the same task graph segment; computing nodes that do not satisfy the chaining conditions are determined as boundary nodes of the task graph segment; the chaining conditions include: the dependency relationship between computing nodes has been determined during the instantiation or update phase, and the dependency relationship can be expressed through chained scheduling information; the device scheduler is specifically configured to: schedule the task description information of each computing node within the same task graph segment sequentially according to the chained scheduling information; for adjacent first and second task graph segments, after the computing nodes in the first task graph segment have completed execution, the device scheduler submits the task description information of the entry computing node of the second task graph segment through a synchronization command.

[0233] According to some embodiments of this disclosure, the partitioning module is specifically used for: obtaining the chain scheduling capability parameters of the target device, where the target device is a parallel computing device for the task graph to be executed, and a device scheduler is set in the target device. The chain scheduling capability parameters include the upper limit of the number of successor indication information that can be configured for a single task description information, and the upper limit of the number of predecessors supported by the device scheduler for multiple predecessor dependencies; in response to the number of successor computing nodes or the number of predecessor computing nodes of a computing node exceeding the corresponding upper limit in the chain scheduling capability parameters, disconnecting the task graph chain segment at the computing node; inserting auxiliary computing tasks between the computing node and its successor computing nodes, so that after the computing node completes its execution, the scheduling of the task description information of the successor computing node is triggered based on the auxiliary computing tasks; or, after the computing node completes its execution, the host submits the task description information of the successor computing node.

[0234] According to some embodiments of this disclosure, the partitioning module is further configured to: in response to the number of consecutive computing nodes within the same task graph segment reaching the number threshold corresponding to the description information cache granularity of the device scheduler, or the computing node being configured with partitioning indication information, disconnect the current task graph segment at the computing node and use the computing node as the entry computing node for a new task graph segment.

[0235] According to some embodiments of this disclosure, the chaining condition further includes: the computing node and its predecessor computing node belong to the same scheduling group; the partitioning module is also used to determine the computing node as the entry computing node of the new task graph chain segment in response to the computing node and its predecessor computing node belonging to different scheduling groups.

[0236] According to some embodiments of this disclosure, the boundary node includes a condition control node. The execution path of the condition control node is determined based on the runtime state after the task graph is started. The partitioning module is specifically used to: partition the computing nodes contained in the condition execution body of the condition control node according to the chaining conditions, and partition the computing nodes that meet the chaining conditions into separate task graph segments; when the condition control node determines to execute the condition execution body, use auxiliary computing tasks to submit the task description information of the entry computing node of the task graph segment corresponding to the condition execution body; when the task graph segment corresponding to the condition execution body is completed, submit the task description information of the subsequent computing nodes through a synchronization command.

[0237] According to some embodiments of this disclosure, during the task graph update phase, the generation module 701 is further configured to: in response to the update result of the startup parameters or resource configuration of the computing node not matching the task description information already generated by the computing node, regenerate the task description information of the computing node and its direct successor computing nodes, and reconfigure the corresponding chain scheduling information; if the task graph has been divided into task graph segments, in response to the change in the chain dependency relationship within the task graph segment caused by the reconfigured chain scheduling information, regenerate the task description information of each computing node located after the computing node in the task graph segment.

[0238] According to some embodiments of this disclosure, it further includes: an insertion module, configured to insert instrumentation auxiliary task description information into the chain dependency relationship formed by the task description information of each computing node before starting the task graph, and configure chain scheduling information for the instrumentation auxiliary task description information; the instrumentation auxiliary task description information is used to statistically analyze the execution information of computing nodes or control the replay of the task graph; after inserting the instrumentation auxiliary task description information, the set of computing nodes in the task graph remains unchanged.

[0239] According to some embodiments of this disclosure, the generation module 701 is specifically used to: store the generated task description information into a description information storage pool accessible on the device side, and the storage address indicated by the subsequent indication information is an address in the description information storage pool.

[0240] According to some embodiments of this disclosure, the device scheduler is further configured to: in response to an error in the execution of a compute node, stop scheduling compute nodes belonging to the same context as the compute node that erred; and in response to a context being reset, resume scheduling of compute nodes within the context.

[0241] According to some embodiments of this disclosure, the device scheduler is further configured to: after each computing node in the task graph chain segment corresponding to the condition executor has completed execution, read the original task description information stored on the device side and restore the task description information of the computing node to a reschedulable state to support multiple executions of the condition executor.

[0242] According to some embodiments of this disclosure, the hardware action indicated by the dependent action information includes any one of the following: a first hardware action, which consists of a scheduling operation, for directly scheduling the task description information of the successor computing node; a second hardware action, which consists of a copy operation and a scheduling operation executed sequentially, for loading the task description information of the successor computing node into the device-side scheduling cache and scheduling the loaded task description information; a third hardware action, which consists of an invalidation operation, a copy operation, and a scheduling operation executed sequentially, for invalidating the old task description information already loaded in the device-side scheduling cache, loading the updated task description information of the successor computing node into the device-side scheduling cache, and scheduling the updated task description information; and a fourth hardware action, which consists of a decrement operation, for decrementing the dependency count value in the task description information of the successor computing node by one.

[0243] The collection, storage, use, processing, transmission, provision, and disclosure of any type of information, such as user personal information, in this technical solution comply with relevant laws and regulations and do not violate public order and good morals.

[0244] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0245] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0246] like Figure 8 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0247] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0248] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the task graph initiation method. For example, in some embodiments, the task graph initiation method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the task graph initiation method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the task graph initiation method by any other suitable means (e.g., by means of firmware).

[0249] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0250] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0251] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0252] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0253] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0254] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0255] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this disclosure can be achieved, and this is not limited herein.

[0256] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A task graph initiation method, comprising: During the instantiation or update phase of the task graph, corresponding task description information is generated for each computing node in the task graph. Based on the dependencies between the computing nodes, chained scheduling information is configured in the task description information, and the chained scheduling information is used at least to indicate the task description information of the successor computing nodes of the computing nodes. When the task graph is initiated, the task description information of the entry computing node of the task graph is submitted so that after the computing node completes its execution, the device scheduler schedules the task description information of the successor computing node according to the chain scheduling information in the task description information of the computing node.

2. The method according to claim 1, wherein, The chained scheduling information includes at least successor indication information and dependent action information. The successor indication information is used to indicate the storage address of the task description information of the successor computing node on the device side. The dependent action information is used to indicate the hardware action that the device scheduler will perform on the task description information of the successor computing node after the computing node completes.

3. The method according to claim 2, wherein, The chained scheduling information also includes an enable flag. The chained scheduling information is configured in the task description information, including: An enable flag is configured in the task description information so that after the computing node completes its execution, the device scheduler accesses the subsequent indication information where the enable flag is valid and skips the subsequent indication information where the enable flag is invalid. The enable flag corresponds one-to-one with the subsequent indication information and is used to indicate whether the corresponding subsequent indication information is valid.

4. The method according to claim 2, wherein, The hardware action consists of at least one of the following operation types: Scheduling operations are used to schedule task description information for subsequent computing nodes. The copy operation is used to load the task description information of the successor computing node into the device-side scheduling cache; The invalidation operation is used to invalidate the old task description information that has been loaded in the device-side scheduling cache; The decrement operation is used to decrement the dependency count value in the task description information of the subsequent computing node by one. The dependency count value is used to record the number of tasks that have not been completed in the predecessor computing nodes of the corresponding computing node.

5. The method according to claim 2 or 4, wherein, The chained scheduling information also includes a dependency count value. Configuring the chained scheduling information in the task description information further includes: For a target computing node with multiple predecessor computing nodes, the dependency count value in the task description information of the target computing node is initialized to the number of predecessor computing nodes of the target computing node. In the task description information of each of the preceding computing nodes, a successor indication information pointing to the task description information of the target computing node is configured. The dependency action information in the task description information of each of the preceding computing nodes is configured to perform a decrement operation on the dependency count value of the target computing node, so that the device scheduler schedules the task description information of the target computing node when the dependency count value of the target computing node is decremented to zero.

6. The method according to claim 2 or 4, wherein, Configure chained scheduling information in the task description information, including: In response to the number of successor computing nodes of the target computing node exceeding the upper limit of the number of successor indications that can be configured for a single task description information, at least one auxiliary task description information is generated, wherein the auxiliary task description information does not correspond to any computing node in the task graph. At least one auxiliary task description information is configured in the chained scheduling information of the task description information of the target computing node; The task description information of at least some of the successor computing nodes of the target computing node is configured in the chained scheduling information of each of the auxiliary task description information, so that the device scheduler schedules each successor computing node of the target computing node according to the chained scheduling information of the auxiliary task description information.

7. The method according to claim 6, wherein, Also includes: In response to the fact that the successor computing node of the target computing node has at least two predecessor computing nodes, the dependency action information in the chained scheduling information of the auxiliary task description information is configured to perform a decrement operation on the dependency count value of the successor computing node.

8. The method according to claim 1, wherein, Generate corresponding task description information for each computing node in the task graph, including: Write the program entry address, grid dimension, thread block dimension, resource configuration information, and constant parameter area address of the corresponding computing node into the task description information.

9. The method according to claim 1, wherein, The chained scheduling information also includes a scheduling trigger flag. Configuring the chained scheduling information in the task description information further includes: Configure a scheduling trigger flag in the task description information. The scheduling trigger flag is used to indicate that the corresponding task description information will not enter the scheduling queue after being loaded to the device side. Before submitting the task description information of the entry computing node of the task graph, the method further includes: preloading the task description information of each computing node other than the entry computing node to the device side; For task description information configured with the aforementioned scheduling trigger flag, the device scheduler performs a scheduling operation on the task description information based on the chained scheduling information of the predecessor computing node corresponding to the task description information, so that the task description information enters the scheduling queue.

10. The method according to claim 9, wherein, The chained scheduling information also includes self-recovery indication information. Configuring the chained scheduling information in the task description information further includes: Configure self-recovery indication information in the task description information of each computing node other than the entry computing node; The self-recovery indication information is used to instruct the device scheduler to read the original task description information stored on the device side after the corresponding computing node has finished executing, and restore the task description information of the computing node to a reschedulable state, so that when the task graph is restarted, the task description information that has been restored to a reschedulable state does not need to be reloaded to the device side.

11. The method according to claim 1, wherein, Also includes: During the instantiation phase, the task description information of each computing node and the startup parameters stored in the constant parameter area corresponding to each task description information are recorded. The constant parameter area is a storage area on the device side used to store the startup parameters of the computing nodes. Before submitting the task description information of the entry computing node of the task graph, the following is also included: Identify the changes that have occurred in this startup compared to the previous startup; In response to changes in startup parameters, the changed startup parameters are written to the corresponding constant parameter area; In response to changes in static fields in the task description information, update the corresponding task description information; For content that has not changed, the content recorded in the instantiation phase is reused.

12. The method according to claim 4, wherein, The task description information for scheduling the successor computing nodes of this computing node includes: If the task description information is updated, the device scheduler performs a scheduling operation on the task description information after performing an invalidation operation and a copy operation on the task description information.

13. The method according to claim 1, wherein, The chained scheduling information also includes prefetching indication information. Configuring the chained scheduling information in the task description information further includes: Configure prefetching indication information in the task description information of the computing node, wherein the prefetching indication information is used to instruct the device scheduler to prefetch the instructions and constant parameters corresponding to the subsequent computing node before scheduling the task description information of the subsequent computing node.

14. The method according to claim 1, wherein, The computation nodes in the task graph include kernel function computation nodes, data copy nodes, and data filling nodes. Corresponding task description information is generated for each computation node in the task graph, including: For data copy nodes and data fill nodes, the data copy operation corresponding to the data copy node or the data fill operation corresponding to the data fill node is converted into a copy operation or fill operation that can be executed on the device side, and corresponding task description information is generated for the converted operation.

15. The method according to claim 1, wherein, The instantiation or update phase also includes: The task graph is divided into at least one task graph chain segment based on the chaining condition. The computing nodes that satisfy the chaining condition are assigned to the same task graph chain segment, and the computing nodes that do not satisfy the chaining condition are determined as the boundary nodes of the task graph chain segment. The chaining condition includes: the dependency relationship between computing nodes has been determined in the instantiation stage or update stage, and the dependency relationship can be expressed through chained scheduling information. The task description information for scheduling the successor computing node of this computing node includes: The device scheduler schedules the task description information of each computing node in the same task graph segment in sequence according to the chained scheduling information. For adjacent first task graph segments and second task graph segments, after the computation node of the first task graph segment has completed its execution, the task description information of the entry computation node of the second task graph segment is submitted through a synchronization command.

16. The method according to claim 15, wherein, The step of dividing the task graph into at least one task graph chain segment based on the chaining condition includes: Obtain the chain scheduling capability parameters of the target device, wherein the target device is a parallel computing device for the task graph to be executed, the device scheduler is set in the target device, and the chain scheduling capability parameters include the upper limit of the number of successor indication information that can be configured for a single task description information, and the upper limit of the number of predecessors supported by the device scheduler for multiple predecessor dependencies. In response to the number of successor computing nodes or predecessor computing nodes of a computing node exceeding the corresponding upper limit in the chain scheduling capability parameter, the task graph chain segment is disconnected at the computing node. An auxiliary computing task is inserted between the computing node and its successor computing node so that after the computing node completes its execution, the scheduling of the task description information of the successor computing node is triggered based on the auxiliary computing task. Alternatively, after the computing node completes its execution, the host submits the task description information of the subsequent computing node.

17. The method according to claim 15, wherein, The step of dividing the task graph into at least one task graph chain segment based on the chaining condition further includes: In response to the number of consecutive computing nodes within the same task graph segment reaching the number threshold corresponding to the description information cache granularity of the device scheduler, or if the computing node is configured with segmentation indication information, the current task graph segment is disconnected at the computing node, and the computing node is used as the entry computing node for a new task graph segment.

18. The method of claim 15, wherein: The chaining condition also includes: the computing node and its predecessor computing node belong to the same scheduling group; In response to the fact that the compute node and its predecessor compute node belong to different scheduling groups, the compute node is identified as the entry compute node of the new task graph segment.

19. The method according to claim 15 or 16, wherein, The boundary nodes include condition control nodes, and the execution path of the condition control nodes is determined based on the runtime state after the task graph is started. Dividing the task graph into at least one task graph segment based on chaining conditions includes: According to the chaining conditions, the computational nodes contained in the condition execution body of the condition control node are divided, and the computational nodes that satisfy the chaining conditions are separately divided into task graph chain segments. When the condition control node determines to execute the condition execution body, the task description information of the entry computing node of the task graph chain segment corresponding to the condition execution body is submitted using the auxiliary computing task; Once the task graph segment corresponding to the condition executor has been executed, the task description information of the subsequent computing nodes is submitted via a synchronization command.

20. The method according to claim 1 or 15, wherein, During the task graph update phase, corresponding task description information is generated for each computing node in the task graph, including: In response to an update result of the startup parameters or resource configuration of the computing node that does not match the task description information already generated by the computing node, the task description information of the computing node and its direct successor computing nodes is regenerated, and the corresponding chain scheduling information is reconfigured. If the task graph has been divided into task graph segments, in response to the change in the chain dependencies within the task graph segments caused by the reconfigured chain scheduling information, the task description information of each computing node located after the computing node in the task graph segment is regenerated.

21. The method according to claim 1, wherein, Also includes: Before starting the task graph, instrumentation auxiliary task description information is inserted into the chain dependency relationship formed by the task description information of each computing node, and chain scheduling information is configured for the instrumentation auxiliary task description information. The instrumentation-assisted task description information is used to statistically analyze the execution information of computing nodes or control the replay of the task graph; after inserting the instrumentation-assisted task description information, the set of computing nodes in the task graph remains unchanged.

22. The method according to claim 2, wherein, The step of generating corresponding task description information for each computing node in the task graph includes: The generated task description information is stored in a description information storage pool accessible on the device side, and the storage address indicated by the subsequent indication information is an address in the description information storage pool.

23. The method according to claim 1, wherein, Also includes: In response to an error in the execution of a compute node, the device scheduler stops scheduling compute nodes that belong to the same context as the compute node that malfunctioned. In response to the context being reset, the device scheduler resumes scheduling of each computing node within the context.

24. The method of claim 19, wherein: After each computing node in the task graph chain segment corresponding to the conditional executor completes its execution, the device scheduler reads the original task description information stored on the device side and restores the task description information of the computing node to a reschedulable state to support multiple executions of the conditional executor.

25. The method according to claim 4, wherein, The hardware action indicated by the action-dependent information includes any one of the following: The first hardware action, which consists of the scheduling operation, is used to directly schedule the task description information of the subsequent computing node. The second hardware action, consisting of the copy operation and the scheduling operation executed sequentially, is used to load the task description information of the subsequent computing node into the device-side scheduling cache and schedule the loaded task description information. The third hardware action, consisting of the invalidation operation, the copy operation, and the scheduling operation executed sequentially, is used to invalidate the old task description information already loaded in the device-side scheduling cache, load the updated task description information of the successor computing node into the device-side scheduling cache, and schedule the updated task description information. The fourth hardware action, which consists of the decrement operation, is used to decrement the dependency count value in the task description information of the subsequent computing node by one.

26. A task map initiation device, comprising: The generation module is used to generate corresponding task description information for each computing node in the task graph during the instantiation or update phase of the task graph. A configuration module is used to configure chained scheduling information in the task description information according to the dependencies between the computing nodes. The chained scheduling information is used to indicate the task description information of the successor computing nodes of the computing nodes. The submission module is used to submit the task description information of the entry computing node of the task graph when the task graph is started, so that the device scheduler can schedule the task description information of the successor computing node according to the chain scheduling information in the task description information of the computing node after the computing node has completed its execution.

27. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-25.

28. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-25.

29. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-25.