A task graph processing apparatus, system, method, and medium

Abstract

This application discloses a task graph processing apparatus, system, method, and medium, relating to the field of data processing technology. The method includes: obtaining modification instructions for the task graph based on a provided application programming interface (API); modifying node parameters of the task graph based on the modification instructions; and incrementally compiling the modified nodes or subgraphs so that the instantiated task graph can be executed. This application provides an API capable of modifying node parameters in a task graph based on modification instructions generated by modification requirements. This solves the technical problem of rigid traditional task graph structures that cannot modify node parameters. Compared to existing technologies that require frequent reconstruction of the task graph, this application can support application scenarios where the computational flow changes frequently during AI training and inference, significantly improving data processing efficiency in multi-stream computing scenarios.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a task graph processing apparatus, system, method, and medium. Background Technology

[0002] In the training and inference of artificial intelligence (AI) models, some computational tasks are organized and scheduled using graphs. In some AI accelerator systems, if a graph scheduling optimization framework is used, user-defined computational tasks are constructed as graph structures to improve task scheduling efficiency and hardware resource utilization. These graph structures are typically instantiated after generation into graph instances (i.e., task graphs) that can be executed by the underlying hardware.

[0003] However, most existing graph management systems employ a static graph mechanism, meaning that once the graph structure is instantiated, it is immediately frozen, and the node parameters of the task graph cannot be modified at runtime. While the static graph mechanism has good execution efficiency in static scenarios where the graph remains unchanged, it has at least the following limitations in applications that require dynamic adjustments to the computation process: if the computation logic changes, the entire task graph must be recompiled and re-instantiated, a process that can take hundreds of milliseconds to several seconds, severely impacting computational efficiency. Summary of the Invention

[0004] The purpose of this application is to provide a task graph processing apparatus, system, method, and medium to solve the technical problem in the prior art that if node parameters change, the entire task graph needs to be reconstructed, which in turn affects computational efficiency.

[0005] To achieve the above objectives, this application provides the following technical solution:

[0006] Firstly, this application proposes a technical solution for a task graph processing apparatus, which includes:

[0007] The instruction receiving module is used to obtain modification instructions for the task graph based on the provided application programming interface; the modification instructions are pre-generated by the processor based on modification requirements; the modification instructions include at least the node information that needs to be modified in the task graph, and the node parameters after the modification.

[0008] The structure modification module is used to modify the structure of the uninstantiated task graph based on the modification instructions, including adding or deleting nodes, dependencies between nodes, and inserting subgraphs;

[0009] The parameter modification module is used to modify the node parameters of the task graph based on the modification instructions.

[0010] The compilation module is used to incrementally compile modified parts of the node or subgraph so that the instantiated task graph can be executed.

[0011] As a specific solution in this application, the application programming interface is a declarative interface, including any one or a combination of kernel node modification interface, memory copy node modification interface, memory setting node modification interface and host setting node modification interface.

[0012] As a specific solution in the technical solution of this application, the kernel node modification interface is used to indicate the modification of the kernel function pointer and / or kernel parameters of the kernel node, wherein the kernel parameters include any one or a combination of input pointers, output pointers and scalar values;

[0013] And / or, the memory copy node modification interface is used to indicate any one or more combinations of the source address, destination address, copy direction, and copy data size of the memory copy node;

[0014] And / or, the memory setting node modification interface is used to indicate any one or more combinations of the target memory address, padding value, and padding data size of the memory setting node;

[0015] And / or, the host setup node modification interface is used to instruct the modification of the central processing unit callback function pointer and / or user-defined data pointer in the host setup node.

[0016] As a specific solution in this application, the task graph includes a main graph and at least one subgraph, wherein the subgraph is nested within the nodes of the main graph; the application programming interface further includes a subgraph modification interface; the subgraph modification interface is used to indicate the modification of the node parameters of the subgraph.

[0017] As a specific solution in this application, the subgraph modification interface is also used to instruct the call to the interface in the application programming interface to modify the node parameters of the subgraph.

[0018] As a specific solution in the technical solution of this application, the parameter modification module is further used to obtain a first subgraph based on the modification instruction; the first subgraph is any subgraph in the task graph whose node parameters need to be modified.

[0019] And, based on the first subgraph, a second subgraph is obtained; the topology of the second subgraph is the same as that of the first subgraph, and the node parameters in the second subgraph are the same as the node parameters in the first subgraph that need to be modified.

[0020] The compilation module is further configured to update the first subgraph in the task graph to the second subgraph based on the subgraph modification interface.

[0021] As a specific solution in this application, the structure modification module is further used to construct a first graph structure based on the first subgraph; the topology of the first graph structure is the same as the topology of the first subgraph; the node parameters of the first graph structure are the same as the node parameters of the first subgraph that need to be modified.

[0022] The compilation module is also used to instantiate the first graph structure and obtain the second subgraph.

[0023] Secondly, this application proposes a technical solution for a task graph processing system, which includes:

[0024] The instruction receiving device is used to obtain modification instructions for the task graph based on the provided application programming interface; the modification instructions are pre-generated by the processor based on modification requirements; the modification instructions include at least the node information that needs to be modified in the task graph, and the node parameters after the modification.

[0025] A structure modification device is used to modify the structure of an uninstantiated task graph based on the modification instructions, including adding or deleting nodes, dependencies between nodes, and inserting subgraphs;

[0026] A parameter modification device is used to modify the node parameters of the task graph based on the modification instruction;

[0027] A compilation apparatus is used to incrementally compile modified parts of the node or subgraph so that the instantiated task graph can be executed.

[0028] As a specific solution in this application, the application programming interface is a declarative interface, including any one or a combination of kernel node modification interface, memory copy node modification interface, memory setting node modification interface and host setting node modification interface.

[0029] As a specific solution in the technical solution of this application, the kernel node modification interface is used to indicate the modification of the kernel function pointer and / or kernel parameters of the kernel node, wherein the kernel parameters include any one or a combination of input pointers, output pointers and scalar values;

[0030] And / or, the memory copy node modification interface is used to indicate any one or more combinations of the source address, destination address, copy direction, and copy data size of the memory copy node;

[0031] And / or, the memory setting node modification interface is used to indicate any one or more combinations of the target memory address, padding value, and padding data size of the memory setting node;

[0032] And / or, the host setup node modification interface is used to instruct the modification of the central processing unit callback function pointer and / or user-defined data pointer in the host setup node.

[0033] As a specific solution in this application, the task graph includes a main graph and at least one subgraph, wherein the subgraph is nested within the nodes of the main graph; the application programming interface further includes a subgraph modification interface; the subgraph modification interface is used to indicate the modification of the node parameters of the subgraph.

[0034] As a specific solution in this application, the subgraph modification interface is also used to instruct the call to the interface in the application programming interface to modify the node parameters of the subgraph.

[0035] As a specific solution in the technical solution of this application, the parameter modification device is further used to obtain a first sub-graph based on the modification instruction; the first sub-graph is any sub-graph in the task graph that requires modification of node parameters;

[0036] And, based on the first subgraph, a second subgraph is obtained; the topology of the second subgraph is the same as that of the first subgraph, and the node parameters in the second subgraph are the same as the node parameters in the first subgraph that need to be modified.

[0037] The compilation device is further configured to update the first subgraph in the task graph to the second subgraph based on the subgraph modification interface.

[0038] As a specific solution in this application, the structure modification device is further used to construct a first graph structure based on the first subgraph; the topology of the first graph structure is the same as the topology of the first subgraph; the node parameters of the first graph structure are the same as the node parameters of the first subgraph that need to be modified.

[0039] The compilation device is further configured to instantiate the first graph structure and obtain the second subgraph.

[0040] Thirdly, this application proposes a technical solution for a task graph processing method, which is applied to a task graph runtime modification scenario. The task graph processing method includes:

[0041] Based on the provided application programming interface, the modification instructions of the task graph are obtained; the modification instructions are pre-generated by the processor based on the modification requirements; the modification instructions include at least the node information of the task graph that needs to be modified, and the node parameters after the modification.

[0042] Based on the modification instructions, the node parameters of the task graph are modified;

[0043] Incremental compilation is performed on the modified nodes or subgraphs so that the instantiated task graph can be executed.

[0044] As a specific solution in this application, the application programming interface is a declarative interface, including any one or a combination of kernel node modification interface, memory copy node modification interface, memory setting node modification interface and host setting node modification interface.

[0045] As a specific solution in the technical solution of this application, the modification of the node parameters of the task graph based on the modification instruction includes: based on the kernel node modification interface, instructing the modification of the kernel function pointer and / or kernel parameters of the kernel node, wherein the kernel parameters include any one or more combinations of input pointers, output pointers and scalar values;

[0046] And / or, based on the memory copy node modification interface, instruct to modify any one or more of the following combinations: source address, destination address, copy direction, and copy data size of the memory copy node;

[0047] And / or, based on the memory setting node modification interface, instruct to modify any one or more of the target memory address, padding value, and padding data size of the memory setting node;

[0048] And / or, based on the host setup node modification interface, instruct the modification of the central processing unit callback function pointer and / or user-defined data pointer in the host setup node.

[0049] As a specific solution in this application, the task graph includes a main graph and at least one subgraph, the subgraph being nested within nodes of the main graph; the application programming interface further includes a subgraph modification interface; the modification of node parameters of the task graph based on the modification instruction further includes: instructing the modification of node parameters of the subgraph based on the subgraph modification interface.

[0050] As a specific solution in this application, the step of instructing to modify the node parameters of the subgraph based on the subgraph modification interface includes: instructing to call an interface in the application programming interface to modify the node parameters of the subgraph based on the subgraph modification interface.

[0051] As a specific solution in the technical solution of this application, the step of instructing to modify the node parameters of the subgraph based on the subgraph modification interface includes: obtaining a first subgraph based on the modification instruction; the first subgraph is any subgraph in the task graph whose node parameters need to be modified.

[0052] Based on the first subgraph, a second subgraph is obtained; the topology of the second subgraph is the same as that of the first subgraph, and the node parameters in the second subgraph are the same as the node parameters in the first subgraph that need to be modified.

[0053] Based on the subgraph modification interface, the first subgraph in the task graph is updated to the second subgraph.

[0054] As a specific solution in this application, obtaining the second subgraph based on the first subgraph includes: constructing a first graph structure based on the first subgraph; the topology of the first graph structure is the same as the topology of the first subgraph; the node parameters of the first graph structure are the same as the node parameters of the first subgraph that need to be modified.

[0055] Instantiate the first graph structure to obtain the second subgraph.

[0056] Fourthly, this application proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the task graph processing method as described in any one of the third aspects.

[0057] Compared with the prior art, the beneficial effects of this application are:

[0058] Based on the modification requirements, this application generates modification instructions and provides an application programming interface that can modify the node parameters in the task graph. This solves the technical problem that the traditional task graph structure is rigid and cannot modify node parameters. Compared with the existing technology that requires frequent reconstruction of the task graph, this application can support application scenarios where the computation process changes frequently during AI training and inference, and significantly improves the data processing efficiency in multi-stream computing scenarios. Attached Figure Description

[0059] Figure 1 This is a flowchart illustrating a task graph processing method proposed in an embodiment of this application.

[0060] Figure 2 This is a schematic diagram of the structure of a task graph processing device proposed in an embodiment of this application;

[0061] Figure 3This is a schematic diagram of the structure of a task graph processing system proposed in an embodiment of this application;

[0062] Figure 4 This is a schematic diagram of a task diagram proposed in an embodiment of this application;

[0063] Figure 5 This is a schematic diagram of another task diagram proposed in the embodiments of this application;

[0064] Figure 6 This is a schematic diagram of the structure of a server proposed in an embodiment of this application. Detailed Implementation

[0065] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects (e.g., the first sub-figure and the second sub-figure are represented as different sub-figures, and so on), and are not necessarily used to describe a specific order or sequence. It should be understood that such names can be used interchangeably where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products, or devices. The division of modules in the embodiments of this application is merely a logical division; in actual applications, there may be other division methods. For example, multiple modules may be combined into or integrated into another system, or some features may be omitted or not performed. Additionally, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interface, indirect coupling between modules, or electrical or other similar forms of communication connection, none of which are limited in the embodiments of this application. Furthermore, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed among multiple circuit modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of this application.

[0066] Before understanding the embodiments of this application, it should be clear that the solutions provided in the embodiments of this application involve technologies such as Artificial Intelligence (AI), Computer Vision (CV), and Machine Learning (ML), which are specifically illustrated through the following embodiments:

[0067] AI, or Artificial Intelligence, refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, Artificial Intelligence is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine capable of reacting in a manner similar to human intelligence. Artificial Intelligence studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0068] AI technology is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0069] Computer vision (CV) is the science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes for target recognition, tracking, and measurement, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include adversarial perturbation generation, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0070] As is known from the background art, during the training and inference of artificial intelligence models, changes in the computational process necessitate frequent reconstruction of the task graph, thus impacting computational efficiency. In the embodiments described below, the task graph processing method proposed in this application is illustrated only using the application scenario of artificial intelligence model training and inference. It does not imply that the task graph processing method proposed in this application is only applicable to the application scenario of artificial intelligence model training and inference. It should be understood that the task graph processing method proposed in this application is applicable to any application scenario requiring frequent reconstruction of the task graph.

[0071] To address the technical problem of frequent task graph reconstruction due to frequent changes in computational processes during AI model training and inference, which impacts computational efficiency, this application proposes an embodiment of a task graph processing method. Specifically, the task graph processing method is as follows: Figure 1 As shown, steps 100 to 200 are included.

[0072] Step 100: Obtain the modification instructions for the task graph based on the provided Application Programming Interface (API).

[0073] In this embodiment, the modification instruction is pre-generated by the processor based on modification requirements. The modification instruction includes the node information in the task graph that needs to be modified, and the modified node parameters.

[0074] In AI model training and inference applications, modification requirements can be designed into the steps of the AI ​​model. When a step with a modification requirement is executed, the processor can generate modification instructions based on the modification requirements. Of course, in some embodiments of this application, the modification requirements can be directly used as modification instructions.

[0075] Step 200: Modify the node parameters of the task graph based on the modification instructions.

[0076] It's important to understand that a task graph is a model that uses a directed acyclic graph (DAG) to represent tasks and their dependencies. It typically contains nodes and directed edges. Nodes represent specific "tasks" (e.g., a function, an operation step, a computational unit), containing information such as the task name, execution logic (e.g., computational operations or processing functions), and input / output parameters. Directed edges represent the "dependencies" between tasks (e.g., ...). Figure 4 As shown, task B (i.e., node B) can only be executed after task A (i.e., node A) is completed (represented by directed edges from task A to task B). The core value of a task graph is to clearly describe the execution order, parallelism, and constraints of tasks, making complex processes (such as data processing and computational tasks) plannable and manageable.

[0077] To facilitate understanding of the task diagram, such as Figure 4 As shown, this application proposes an embodiment of a task graph. The task graph contains five nodes: node A, node B, node C, node D, and node E. Their dependencies are as follows:

[0078] Node A is the starting node;

[0079] Nodes B and C depend on node A;

[0080] After nodes B and C complete their execution, they together act as a dependent trigger node D.

[0081] After node D completes its execution, node E will be scheduled for execution.

[0082] exist Figure 4 In the diagram, each circle represents a node, and each arrow represents a directed edge.

[0083] It should be clear that in this application, only the following is used: Figure 4 The task diagram shown is an example illustrating the structure of a task diagram, and does not imply that the task diagram processing method proposed in this application is only applicable to tasks such as... Figure 4 The task graph shown is provided. It should be understood that the task graph processing method proposed in this application is applicable to task graphs of any form.

[0084] It's important to understand that an API is a predefined set of functions, methods, or protocols. A task graph API is specifically a collection of interfaces used to define, manipulate, execute, and monitor task graphs. Simply put, the purpose of an API is to allow developers to quickly manage the entire lifecycle of a task graph by calling readily available tools provided by the API, without having to build the parsing, scheduling, and execution logic from scratch.

[0085] Before understanding the various application programming interfaces that support dynamic updates of the task graph, it is important to understand that the steps for building the task graph include steps S1 to S3.

[0086] Step S1: Construct the graph structure.

[0087] It is important to understand that during the graph structure construction phase, various types of nodes are typically provided to represent different computational and data operation needs.

[0088] Specifically, the nodes in the graph structure include, but are not limited to:

[0089] Kernel Node, Memcpy Node, MemsetNode, Event Wait Node, Event Record Node, Memory Allocation Node, Memory Free Node, and ChildGraph Node.

[0090] In the task graph, a kernel node represents a core computational operation that needs to be executed. It is usually mapped to a specific computational function on a graphics processing unit (GPU), central processing unit (CPU), or other accelerator. In other words, kernel nodes are used to execute specific computational tasks, such as matrix multiplication, convolution operations, activation functions, and other core algorithms.

[0091] In the task graph, a memory copy node is responsible for copying data between different memory regions. For example, it's a node that copies data from host memory to device memory, or a node that transfers data between different memory spaces within a device. Data transfer is a critical aspect of heterogeneous computing, and memory copy nodes ensure the correct flow of data between different computing units and memory levels.

[0092] In the task graph, the memory setup node is used to initialize memory regions, setting specified memory blocks to specific values ​​(e.g., 0 or 1). That is, the memory setup node is used to prepare the initial state before computation begins, such as initializing output buffers or intermediate result storage areas.

[0093] In the task graph, event wait nodes are nodes used for synchronization operations, pausing the current execution flow until a specific event completes. In other words, event wait nodes ensure that subsequent operations only begin after the dependent event (e.g., data transfer or computation task) has completed, and are a key mechanism for achieving precise synchronization.

[0094] Event logging nodes are defined in the task graph as nodes that record specific points in time or the completion of operations, used to mark the occurrence of events. Event logging nodes are generally used in conjunction with event waiting nodes to create synchronization points, and can also be used for performance analysis, measuring operation execution time, etc.

[0095] In the task graph, a memory allocation node is responsible for allocating space in memory, preparing memory resources for subsequent data storage and operations. In other words, memory allocation nodes dynamically manage memory resources, ensuring sufficient space to store inputs, outputs, and intermediate results during computation.

[0096] In the task graph, a memory release node is a node that releases memory space that is no longer needed, reclaiming memory resources for other operations. In other words, memory release nodes are used to prevent memory leaks and optimize memory usage efficiency, especially in long-running or resource-constrained environments.

[0097] Subgraph nodes in a task graph represent nodes nested within sub-computation graphs, breaking down complex operational processes into smaller subgraph flows. Subgraph nodes are used to implement hierarchical organization of the task graph, improving maintainability and reusability.

[0098] During the construction of the graph structure, developers can use the Graph Create Node interface, Graph Destroy Node interface, Add Dependencies interface, and Remove Dependencies interface to add nodes of the above types in sequence according to the task logic, and form the graph structure by adding dependencies (i.e. directed edges) between nodes in the task graph.

[0099] Step S2: Modify the graph structure before instantiating it.

[0100] It's important to understand that arbitrary structural modifications to the task graph are allowed before the graph structure is instantiated. These modifications include, but are not limited to: adding or deleting nodes of any type; modifying the node parameters of existing nodes (e.g., memory size, copy direction, kernel configuration, etc.); adding or deleting dependencies between nodes (i.e., directed edges); supporting repeated graph structure validation to ensure that the modified task graph still maintains directed acyclicity; allowing the insertion of subgraph nodes as nodes in the main graph; and nesting a subgraph structure, etc.

[0101] Step S3: Instantiate the graph structure.

[0102] After the graph is constructed, it is instantiated by calling the Deep Learning ContainerGraph Instantiate interface. The instantiation process includes: performing topological sorting based on the dependencies between nodes; dividing nodes that can be executed in parallel into the same scheduling layer; and finally generating a hierarchical directed acyclic graph to support parallel scheduling and optimized execution.

[0103] It is important to understand that in existing technologies, the instantiated graph structure (i.e., the task graph) no longer allows structural changes to its nodes and dependencies, and the node parameters in each node cannot be modified.

[0104] In this embodiment, before the graph structure is instantiated, the structure of the task graph can be arbitrarily modified based on the modification instruction. Specifically, to modify the structure of the uninstantiated task graph based on the modification instruction, refer to step S2 above.

[0105] In order to enable modification of node parameters in the instantiated graph structure (i.e., the task graph), in the embodiments of this application, the application programming interface in step 100 is at least used to modify the node parameters of the task graph. In other words, in this embodiment, the application programming interface can be any interface capable of modifying the node parameters of the instantiated task graph. For example, the application programming interface can be a declarative interface. Declarative interfaces include any one or a combination of kernel node modification interfaces, memory copy node modification interfaces, memory setting node modification interfaces, and host setting node modification interfaces.

[0106] Step 200: Modify the node parameters of the task graph based on the modification instructions.

[0107] In the embodiments of this application, the node parameters of the task graph can be modified in any reasonable manner based on the modification instructions. For example, step 200, modifying the node parameters of the task graph based on the modification instructions, includes any one or a combination of steps 210 to 240.

[0108] Step 210: Modify the kernel function pointer and / or kernel parameters of the kernel node based on the kernel node modification interface instruction. The kernel parameters include any one or more combinations of input pointers, output pointers, and scalar values.

[0109] Step 220: Modify any one or more of the following combinations of the source address, destination address, copy direction, and copy data size of the memory copy node based on the interface instruction of the memory copy node modification.

[0110] Step 230: Modify any one or more of the target memory address, padding value, and padding data size of the memory setting node based on the interface instruction of the memory setting node modification.

[0111] Step 240: Modify the CPU callback function pointer and / or user-defined data pointer in the host setup node based on the host setup node modification interface instruction.

[0112] It is important to understand that in the embodiments of this application, steps 210 and 220 do not represent the order in which the steps are executed; they are merely used to distinguish different steps. For example, in the embodiments of this application, step 210 can be executed first, followed by step 220; step 220 can be executed first, followed by step 210; steps 210 and 220 can be executed simultaneously; only step 210 can be executed without step 220; and only step 220 can be executed without step 210. The same applies to the sequence numbers of other steps in the embodiments of this application, and will not be elaborated upon further.

[0113] The task graph processing method proposed in this embodiment provides an application programming interface that can modify node parameters in the task graph based on the modification instructions generated by the modification requirements. This solves the technical problem that the traditional task graph structure is rigid and cannot modify node parameters. Compared with the existing technology that requires frequent reconstruction of the task graph, this application can support application scenarios where the computation process changes frequently during AI training and inference, and significantly improve the data processing efficiency in multi-stream computing scenarios.

[0114] In some application scenarios, a task graph includes a main graph and at least one subgraph, which is nested within nodes of the main graph. In existing technologies, if modifications to the node parameters of a subgraph are required, the entire task graph still needs to be recompiled and instantiated according to steps S1 to S3.

[0115] To facilitate understanding of the structural relationship between the main graph and subgraphs, such as Figure 5 As shown, this application proposes another embodiment of the task diagram. Figure 5 This illustrates a task graph structure containing subgraphs, consisting of a main graph and nested subgraphs. The main graph contains five nodes: A, B, C, D, and E, with the following dependencies:

[0116] Node A is the starting node;

[0117] Nodes B and C depend on node A;

[0118] After nodes B and C complete their execution, they together act as a dependent trigger node D.

[0119] After node D completes its execution, node E will be scheduled for execution.

[0120] like Figure 5 As shown, node C is not a regular computation node, but rather an encapsulated subgraph node. Subgraph node C contains three nodes: C1, C2, and C3. The task execution dependencies are as follows:

[0121] Node C1 is the starting node of the subgraph;

[0122] Nodes C2 and C3 both depend on node C1 and execute in parallel after C1 has finished executing.

[0123] The completion of the entire subgraph is marked by the completion of both C2 and C3.

[0124] like Figure 5 The scheduling process of the task graph shown is as follows:

[0125] The main graph execution begins from node A;

[0126] After node A completes its task, two branches are scheduled in parallel: node B in the main graph and node C in the nested subgraph.

[0127] The subgraph executes node C1 first; then nodes C2 and C3 are executed in parallel; after the entire subgraph is executed, node C in the main graph is considered complete.

[0128] After nodes B and C in the main graph have completed their execution, node D will be scheduled for execution.

[0129] After node D completes execution, node E will continue to be executed.

[0130] It's important to understand that in task graph design, the main graph and subgraphs represent a hierarchical organizational relationship used to break down complex tasks into smaller, more manageable components. This relationship is similar to the main function and sub-functions in a program, or the parent and child classes in object-oriented programming. In this application, only the following example is used... Figure 5 The task graph shown is an example of a task graph with subgraphs and does not imply that the task graph processing method proposed in this application is only applicable to such task graphs. Figure 5 The task graph shown is provided. It should be understood that the task graph processing method proposed in this application is applicable to task graphs with subgraphs of any form.

[0131] In order to enable modification of node parameters in the subgraph, in one embodiment of this application, the application programming interface further includes a subgraph modification interface. Step 200, based on the modification instruction, modifies the node parameters of the task graph, and further includes step 250.

[0132] Step 250: Based on the subgraph modification interface, instruct the modification of the node parameters of the subgraph.

[0133] In this embodiment, the node parameters of the subgraph can be modified based on the subgraph modification interface in any reasonable manner. For example, the node parameters of the subgraph can be modified as shown in Embodiment 1 or Embodiment 2 below.

[0134] Example 1 of modifying subgraph node parameters

[0135] In this embodiment, step 250, based on the subgraph modification interface, instructs to modify the node parameters of the subgraph, including step 251.

[0136] Step 251: Based on the subgraph modification interface, instruct the call to the interfaces mentioned above (e.g., kernel node modification interface, memory copy node modification interface, memory setting node modification interface, and host setting node modification interface) to modify the node parameters of the subgraph.

[0137] It is important to understand that in the computer field, instructing another interface (i.e., kernel node modification interface, memory copy node modification interface, or memory setting node modification interface, etc.) through a certain interface (i.e., subgraph modification interface) is a mature technology, which will not be elaborated here.

[0138] Example 2 of modifying subgraph node parameters

[0139] In this embodiment, step 250, based on the subgraph modification interface, instructs to modify the node parameters of the subgraph, including steps 252 to 254.

[0140] Step 252: Obtain the first sub-graph based on the modification instructions.

[0141] In this embodiment, the first subgraph is any subgraph in the task graph whose node parameters need to be modified.

[0142] Step 253: Obtain the second subgraph based on the first subgraph.

[0143] In this embodiment, the topology of the second subgraph is the same as that of the first subgraph, and the node parameters in the second subgraph are the same as the modified node parameters in the first subgraph.

[0144] In the embodiments of this application, the second subgraph can be obtained based on the first subgraph in any reasonable manner. For example, before the first subgraph is instantiated, the node parameters in the first subgraph can be directly modified to obtain the second subgraph; or, after the first subgraph is instantiated, step 253, obtaining the second subgraph based on the first subgraph, includes steps 253a and 253b.

[0145] Step 253a: Construct the first graph structure based on the first subgraph.

[0146] In this embodiment, the topology of the first graph structure is the same as that of the first subgraph. The node parameters of the first graph structure are the same as the modified node parameters of the first subgraph.

[0147] Step 253b: Instantiate the first graph structure to obtain the second subgraph.

[0148] In other words, in this embodiment, the second subgraph is reconstructed and generated based on the topology of the first subgraph.

[0149] Step 254: Based on the subgraph modification interface, update the first subgraph in the task graph to the second subgraph.

[0150] It is important to understand that updating one subgraph of a task graph to another without changing the overall topology of the task graph is a mature technique, which will not be elaborated here.

[0151] This concludes the description of Example 2 for modifying subgraph node parameters.

[0152] Step 300: Incrementally compile the modified nodes or subgraphs so that the instantiated task graph can be executed.

[0153] The embodiments of the task graph processing method proposed in this application provide an application programming interface that can modify the node parameters in the task graph based on the modification instructions generated by the modification requirements. This solves the technical problem that the traditional task graph structure is rigid and cannot modify the node parameters. Compared with the prior art, which requires frequent reconstruction of the task graph, this application can support application scenarios where the computation process changes frequently during AI training and inference, and significantly improve the data processing efficiency in multi-stream computing scenarios.

[0154] Having introduced the task graph processing method proposed in the embodiments of this application, the following describes a task graph processing apparatus proposed in the embodiments of this application, such as... Figure 2 As shown, the task graph processing device 10 includes:

[0155] The instruction receiving module 11 is used to obtain the modification instructions of the task graph based on the provided application programming interface; the modification instructions are pre-generated by the processor based on the modification requirements; the modification instructions include at least the node information of the task graph that needs to be modified, and the node parameters of the modified nodes;

[0156] The structure modification module 12 is used to modify the structure of the uninstantiated task graph based on the modification instructions, including adding or deleting nodes, dependencies between nodes, and inserting subgraphs.

[0157] The parameter modification module 13 is used to modify the node parameters of the task graph based on the modification instruction;

[0158] Compilation module 14 is used to incrementally compile modified parts of the node or subgraph so that the instantiated task graph can be executed.

[0159] As a specific embodiment of this application, the application programming interface is a declarative interface, including any one or a combination of multiple of the kernel node modification interface, memory copy node modification interface, memory setting node modification interface, and host setting node modification interface.

[0160] As a specific embodiment of this application, the kernel node modification interface is used to indicate the modification of the kernel function pointer and / or kernel parameters of the kernel node, wherein the kernel parameters include any one or a combination of input pointers, output pointers and scalar values;

[0161] And / or, the memory copy node modification interface is used to indicate any one or more combinations of the source address, destination address, copy direction, and copy data size of the memory copy node;

[0162] And / or, the memory setting node modification interface is used to indicate any one or more combinations of the target memory address, padding value, and padding data size of the memory setting node;

[0163] And / or, the host setup node modification interface is used to instruct the modification of the central processing unit callback function pointer and / or user-defined data pointer in the host setup node.

[0164] As a specific embodiment of this application, the task graph includes a main graph and at least one subgraph, the subgraph being nested within nodes of the main graph; the application programming interface further includes a subgraph modification interface; the subgraph modification interface is used to indicate the modification of node parameters of the subgraph.

[0165] As a specific embodiment of this application, the subgraph modification interface is also used to instruct the calling of the interface in the application programming interface to modify the node parameters of the subgraph.

[0166] As a specific embodiment of this application, the parameter modification module 13 is further configured to obtain a first subgraph based on the modification instruction; the first subgraph is any subgraph in the task graph whose node parameters need to be modified.

[0167] And, based on the first subgraph, a second subgraph is obtained; the topology of the second subgraph is the same as that of the first subgraph, and the node parameters in the second subgraph are the same as the node parameters in the first subgraph that need to be modified.

[0168] The compilation module 14 is further configured to update the first subgraph in the task graph to the second subgraph based on the subgraph modification interface.

[0169] As a specific embodiment of this application, the structure modification module 12 is further configured to construct a first graph structure based on the first subgraph; the topology of the first graph structure is the same as the topology of the first subgraph; the node parameters of the first graph structure are the same as the node parameters of the first subgraph that need to be modified.

[0170] The compilation module 14 is further configured to instantiate the first graph structure and obtain the second subgraph.

[0171] The embodiments of the task graph processing device proposed in this application provide an application programming interface that can modify the node parameters in the task graph based on the modification instructions generated by the modification requirements. This solves the technical problem that the traditional task graph structure is rigid and cannot modify the node parameters. Compared with the prior art, which requires frequent reconstruction of the task graph, this application can support application scenarios where the computation process changes frequently during AI training and inference, and significantly improve the data processing efficiency in multi-stream computing scenarios.

[0172] Having introduced the task graph processing apparatus proposed in the embodiments of this application, the following describes a task graph processing system proposed in the embodiments of this application, such as... Figure 3 As shown, the task graph processing system 20 includes:

[0173] The instruction receiving device 21 is used to obtain modification instructions for the task graph based on the provided application programming interface; the modification instructions are pre-generated by the processor based on modification requirements; the modification instructions include at least the node information that needs to be modified in the task graph, and the node parameters after the modification.

[0174] The structure modification device 22 is used to modify the structure of an uninstantiated task graph based on the modification instructions, including adding or deleting nodes, dependencies between nodes, and inserting subgraphs.

[0175] The parameter modification device 23 is used to modify the node parameters of the task graph based on the modification instruction;

[0176] The compilation unit 24 is used to incrementally compile the modified partial nodes or subgraphs so that the instantiated task graph can be executed.

[0177] As a specific embodiment of this application, the application programming interface is a declarative interface, including any one or a combination of multiple of the kernel node modification interface, memory copy node modification interface, memory setting node modification interface, and host setting node modification interface.

[0178] As a specific embodiment of this application, the kernel node modification interface is used to indicate the modification of the kernel function pointer and / or kernel parameters of the kernel node, wherein the kernel parameters include any one or a combination of input pointers, output pointers and scalar values;

[0179] And / or, the memory copy node modification interface is used to indicate any one or more combinations of the source address, destination address, copy direction, and copy data size of the memory copy node;

[0180] And / or, the memory setting node modification interface is used to indicate any one or more combinations of the target memory address, padding value, and padding data size of the memory setting node;

[0181] And / or, the host setup node modification interface is used to instruct the modification of the central processing unit callback function pointer and / or user-defined data pointer in the host setup node.

[0182] As a specific embodiment of this application, the task graph includes a main graph and at least one subgraph, the subgraph being nested within nodes of the main graph; the application programming interface further includes a subgraph modification interface; the subgraph modification interface is used to indicate the modification of node parameters of the subgraph.

[0183] As a specific embodiment of this application, the subgraph modification interface is also used to instruct the calling of the interface in the application programming interface to modify the node parameters of the subgraph.

[0184] As a specific embodiment of this application, the parameter modification device 23 is further configured to obtain a first subgraph based on the modification instruction; the first subgraph is any subgraph in the task graph whose node parameters need to be modified.

[0185] And, based on the first subgraph, a second subgraph is obtained; the topology of the second subgraph is the same as that of the first subgraph, and the node parameters in the second subgraph are the same as the node parameters in the first subgraph that need to be modified.

[0186] The compilation device 24 is further configured to update the first subgraph in the task graph to the second subgraph based on the subgraph modification interface.

[0187] As a specific embodiment of this application, the structure modification device 22 is further configured to construct a first graph structure based on the first subgraph; the topology of the first graph structure is the same as the topology of the first subgraph; the node parameters of the first graph structure are the same as the node parameters of the first subgraph that need to be modified.

[0188] The compilation device 24 is further configured to instantiate the first graph structure and obtain the second subgraph.

[0189] The embodiments of the task graph processing system proposed in this application provide an application programming interface that can modify the node parameters in the task graph based on the modification instructions generated by the modification requirements. This solves the technical problem that the traditional task graph structure is rigid and cannot modify the node parameters. Compared with the prior art, which requires frequent reconstruction of the task graph, this application can support application scenarios where the computation process changes frequently during AI training and inference, and significantly improve the data processing efficiency in multi-stream computing scenarios.

[0190] Having introduced the task graph processing system proposed in the embodiments of this application, the following describes a computer-readable storage medium proposed in the embodiments of this application. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the task graph processing method as described in any of the above embodiments.

[0191] In this embodiment, the computer-readable storage medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0192] Having described the computer-readable storage medium proposed in the embodiments of this application, a computing device proposed in this application is now described. This computing device includes a processor, on which a computer program is stored. When executed, the computer program implements the task graph processing method as described in any of the above embodiments.

[0193] It should be noted that the computing devices involved in the embodiments of this application may be servers and / or terminal devices.

[0194] The server involved in the embodiments of this application can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0195] The terminal devices involved in the embodiments of this application can be devices that provide voice and / or data connectivity to users, handheld devices with wireless connectivity, or other processing devices connected to a wireless modem. Examples include mobile phones (or "cellular" phones) and computers with mobile terminals, such as portable, pocket-sized, handheld, computer-embedded, or vehicle-mounted mobile devices that exchange voice and / or data with a wireless access network. Examples include Personal Communication Service (PCS) phones, cordless phones, Session Initiation Protocol (SIP) phones, Wireless Local Loop (WLL) stations, Personal Digital Assistants (PDAs), and other devices.

[0196] In this application embodiment, the processor is an integrated circuit chip with signal processing capabilities. For example, the processor may be an FPGA, a general-purpose processor, a DSP, an ASIC, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, a SoC, a CPU, a network processor (NP), a microcontroller unit (MCU), a PLD, or other integrated chips, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in this application embodiment. In one possible implementation, this application embodiment also provides a computer-readable storage medium storing program code, which, when executed on the computer, causes the computer to perform the above-described method embodiments.

[0197] This application also provides a terminal device, such as... Figure 6 As shown, for ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. The terminal device can be any terminal device including mobile phones, tablets, personal digital assistants (PDAs), point-of-sale (POS) terminals, in-vehicle computers, servers, etc. Taking a server as an example:

[0198] Please see Figure 6 , Figure 6This is a schematic diagram of a server structure provided in an embodiment of this application. The server 1100 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1122 (e.g., one or more processors) and memory 1132, and one or more storage media 1130 (e.g., one or more mass storage devices) for storing application programs 1142 or data 1144. The memory 1132 and storage media 1130 may be temporary or persistent storage. The program stored in the storage media 1130 may include one or more modules (not shown in the figure), each module may include a series of instruction operations on the server. Furthermore, the CPU 1122 may be configured to communicate with the storage media 1130 and execute the series of instruction operations in the storage media 1130 on the server 1100.

[0199] Server 1100 may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input / output interfaces 1158, and / or one or more operating systems 1141, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

[0200] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0201] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0202] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, apparatuses, or modules, and may be electrical, mechanical, or other forms.

[0203] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0204] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0205] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0206] The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0207] The technical solutions provided in the embodiments of this application have been described in detail above. Specific examples have been used in the embodiments of this application to illustrate the principles and implementation methods of the embodiments of this application. The description of the above embodiments is only for the purpose of helping to understand the methods and core ideas of the embodiments of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the embodiments of this application. Therefore, the content of this specification should not be construed as a limitation on the embodiments of this application.

Claims

1. A task graph processing apparatus, characterized in that, include: The instruction receiving module is used to obtain modification instructions for the task graph based on the provided application programming interface; the modification instructions are pre-generated by the processor based on modification requirements; the modification instructions include at least the node information that needs to be modified in the task graph, and the node parameters after the modification. The structure modification module is used to modify the structure of the uninstantiated task graph based on the modification instructions, including adding or deleting nodes, dependencies between nodes, and inserting subgraphs; The parameter modification module is used to modify the node parameters of the task graph based on the modification instructions. The compilation module is used to incrementally compile modified parts of the node or subgraph so that the instantiated task graph can be executed; The task graph includes a main graph and at least one subgraph, the subgraph being nested within nodes of the main graph; the application programming interface further includes a subgraph modification interface; the subgraph modification interface is used to indicate the modification of node parameters of the subgraph; The parameter modification module is further configured to obtain a first subgraph based on the modification instruction; the first subgraph is any subgraph in the task graph whose node parameters need to be modified. And, based on the first subgraph, a second subgraph is obtained; the topology of the second subgraph is the same as that of the first subgraph, and the node parameters in the second subgraph are the same as the node parameters in the first subgraph that need to be modified. The compilation module is further configured to update the first subgraph in the task graph to the second subgraph based on the subgraph modification interface.

2. The task graph processing apparatus according to claim 1, characterized in that, The application programming interface is a declarative interface, including any one or a combination of kernel node modification interface, memory copy node modification interface, memory setting node modification interface and host setting node modification interface.

3. The task graph processing apparatus according to claim 2, characterized in that, The kernel node modification interface is used to indicate the modification of kernel function pointers and / or kernel parameters of the kernel node, wherein the kernel parameters include any one or a combination of input pointers, output pointers and scalar values; And / or, the memory copy node modification interface is used to indicate any one or more combinations of the source address, destination address, copy direction, and copy data size of the memory copy node; And / or, the memory setting node modification interface is used to indicate any one or more combinations of the target memory address, padding value, and padding data size of the memory setting node; And / or, the host setup node modification interface is used to instruct the modification of the central processing unit callback function pointer and / or user-defined data pointer in the host setup node.

4. The task graph processing apparatus according to claim 1, characterized in that, The subgraph modification interface is also used to instruct the call to an interface in the application programming interface to modify the node parameters of the subgraph.

5. The task graph processing apparatus according to claim 1, characterized in that, The structure modification module is further configured to construct a first graph structure based on the first subgraph; the topology of the first graph structure is the same as the topology of the first subgraph; the node parameters of the first graph structure are the same as the node parameters of the first subgraph after modification. The compilation module is also used to instantiate the first graph structure and obtain the second subgraph.

6. A task graph processing system, characterized in that, include: The instruction receiving device is used to obtain modification instructions for the task graph based on the provided application programming interface; the modification instructions are pre-generated by the processor based on modification requirements; the modification instructions include at least the node information that needs to be modified in the task graph, and the node parameters after the modification. A structure modification device is used to modify the structure of an uninstantiated task graph based on the modification instructions, including adding or deleting nodes, dependencies between nodes, and inserting subgraphs; A parameter modification device is used to modify the node parameters of the task graph based on the modification instruction; A compilation device is used to incrementally compile modified parts of nodes or subgraphs so that the instantiated task graph can be executed; The task graph includes a main graph and at least one subgraph, the subgraph being nested within nodes of the main graph; the application programming interface further includes a subgraph modification interface; the subgraph modification interface is used to indicate the modification of node parameters of the subgraph; The parameter modification device is further configured to obtain a first subgraph based on the modification instruction; the first subgraph is any subgraph in the task graph whose node parameters need to be modified. And, based on the first subgraph, a second subgraph is obtained; the topology of the second subgraph is the same as that of the first subgraph, and the node parameters in the second subgraph are the same as the node parameters in the first subgraph that need to be modified. The compilation device is further configured to update the first subgraph in the task graph to the second subgraph based on the subgraph modification interface.

7. A task graph processing method, characterized in that, This method, applied to scenarios involving runtime modification of the task graph, includes: Based on the provided application programming interface, the modification instructions of the task graph are obtained; the modification instructions are pre-generated by the processor based on the modification requirements; the modification instructions include at least the node information of the task graph that needs to be modified, and the node parameters after the modification. Based on the modification instructions, the node parameters of the task graph are modified; Incremental compilation is performed on the modified nodes or subgraphs so that the instantiated task graph can be executed; The task graph includes a main graph and at least one subgraph, the subgraph being nested within nodes of the main graph; the application programming interface further includes a subgraph modification interface; the subgraph modification interface is used to indicate the modification of node parameters of the subgraph; The method of modifying the subgraph based on the subgraph modification interface, instructing to modify the node parameters of the subgraph, includes: obtaining a first subgraph based on the modification instruction; the first subgraph is any subgraph in the task graph whose node parameters need to be modified. Based on the first subgraph, a second subgraph is obtained; the topology of the second subgraph is the same as that of the first subgraph, and the node parameters in the second subgraph are the same as the node parameters in the first subgraph that need to be modified. Based on the subgraph modification interface, the first subgraph in the task graph is updated to the second subgraph.

8. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a processor, implements the task graph processing method as described in claim 7.

Images