A dynamic cloud scheduling method and system for artificial intelligence workflow

Abstract

The present application relates to the technical field of computer, specifically relates to a kind of dynamic cloud scheduling method and system of artificial intelligence workflow;The method of the present application includes defining standardized AI-Workflow IR intermediate representation, convert native workflow into the workflow of standardized AI-Workflow IR intermediate representation, build calculation dependency graph, obtain the key path node in calculation dependency graph by CPA algorithm, form key path, mark the subgraph in key path, carry out subgraph segmentation and subgraph optimization to the workflow of standardized AI-Workflow IR intermediate representation using GRR algorithm, based on deep reinforcement learning algorithm, the matching scheduling of Agent is carried out to subgraph;The present application supports the conversion of workflow of different frameworks, optimizes scheduling strategy with environmental change, JIT-Agent generates specialized service on demand, and realizes the distribution of workflow reasonably.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and specifically to a dynamic cloud scheduling method and system for artificial intelligence workflows. Background Technology

[0002] Current AI workflow execution and scheduling solutions, whether running locally on a single machine or deployed in the traditional cloud, all suffer from the following core shortcomings:

[0003] Single-machine execution bottleneck: Traditional node editors rely entirely on local hardware resources. Insufficient GPU memory and CPU computing power result in low execution efficiency for large models, or even failure to run them; and resources cannot be flexibly expanded according to the workload.

[0004] Manual API integration is costly: Existing cloud solutions require users to manually modify workflow logic and client code, which disrupts the integrity of the original workflow and requires users to have cloud development capabilities, making it a high barrier to entry.

[0005] The architecture is heavily coupled: the scheduling logic and execution logic are tightly bound together (such as fixed subgraph allocation rules and static server mapping). When the types of workflow nodes increase or the status of cloud resources changes, the system needs to be restructured as a whole, and the complexity increases exponentially.

[0006] The optimization methods are limited and passive: they can only optimize execution efficiency through simple means such as data compression and task sharding, lacking proactive analysis (such as dependencies and operator characteristics) and in-depth optimization (such as subgraph rewriting and redundancy elimination) of the workflow computation graph, resulting in serious resource waste (such as repeated model loading and invalid data transmission).

[0007] Microservice maintenance is difficult: Traditional cloud solutions use large and comprehensive container images (containing all models and nodes), resulting in huge image sizes (often tens of GB), frequent dependency conflicts (such as different models having different requirements for CUDA versions), and the inability to dynamically adjust service capabilities based on request characteristics.

[0008] Patent CN120634156A proposes a cloud scheduling scheme based on subgraph segmentation and static load balancing using serialization attributes, but its core design still has the following key limitations, failing to meet the complex requirements of AI workflows:

[0009] Subgraph segmentation is achieved solely through the serialization attributes of native leaf nodes, without establishing a standardized mathematical model for the workflow. When the native workflow comes from different frameworks (such as ComfyUI and Blender), incompatibility in node types and data formats leads to the need for repeated development of segmentation logic for different frameworks, and cross-framework unified optimization is not possible.

[0010] The splitting criterion is solely whether the data is serializable, without considering the computational characteristics and dependencies of the workflow. For example, forcibly separating the model loading node from the image encoding node leads to repeated model loading, which actually increases execution time.

[0011] The matching strategy based on load information only focuses on the current load of the cloud server and ignores the matching degree between the subgraph and the server's capabilities; moreover, the scheduling decision has no dynamic learning ability and cannot adapt to long-term changes in request characteristics.

[0012] Relying on pre-deployed fixed server clusters, when new types of workflows or high-frequency request subgraphs emerge, it is unable to dynamically generate microservice agents specializing in the task, resulting in low resource utilization and high response latency. Summary of the Invention

[0013] To address the shortcomings of the existing technologies, this invention aims to provide a dynamic cloud scheduling method and system for artificial intelligence workflows, enabling the rational allocation of agents in workflow subgraphs.

[0014] To solve the above problems, the present invention adopts the following technical solution:

[0015] On one hand, the present invention provides a dynamic cloud scheduling method for artificial intelligence workflows, comprising:

[0016] Define a standardized AI-Workflow IR intermediate representation to convert native workflows into workflows represented by the standardized AI-Workflow IR intermediate representation;

[0017] Based on the standardized AI-Workflow IR intermediate representation of the workflow, a computational dependency graph is constructed. The critical path nodes in the computational dependency graph are obtained through the CPA algorithm, forming the critical path. The subgraphs in the critical path are then marked.

[0018] The GRR algorithm is used to perform subgraph segmentation and subgraph optimization on the standardized AI-Workflow IR intermediate representation of the workflow;

[0019] Agent matching and scheduling are performed on subgraphs based on deep reinforcement learning algorithms.

[0020] As one possible implementation, the process of converting the native workflow into a standardized AI-Workflow IR intermediate representation includes:

[0021] Parse the original workflow, extract the node set, edge set, and attribute set of the original workflow, and extract the node type, input parameters, and metadata for each node;

[0022] Iterate through the input parameters of each node, identify static parameters, generate operators for static parameters, and add static constants to the constant set of the AI-Workflow IR module;

[0023] Based on the relationship between native nodes and IR operators of the AI-Workflow IR module, each node is mapped to an IR operator based on its node type;

[0024] Traverse the edge set of the native workflow, identify whether the input parameter of a node is a node reference. If so, use the output of the IR operator of the parent node of the node reference as the input of the IR operator of the lower node. If the input parameter of the node is identified as a static parameter, use the static constant corresponding to the node as the input of the IR operator.

[0025] Based on data dependencies, all operators are written into the @main function, metadata is added to each operator, and a complete standardized AI-Workflow IR intermediate representation is generated.

[0026] As one possible implementation, the construction of the computation dependency graph, which uses the CPA algorithm to derive the critical path nodes in the computation dependency graph, and forms the critical path includes:

[0027] Based on the @main function of the standardized AI-Workflow IR intermediate representation, a computational dependency graph containing a set of nodes, a set of edges, and a set of weights is constructed. The nodes in the node set represent an operator, the edges in the edge set represent the data dependencies between operators, and the weights in the weight set represent the execution time of the operator on standard hardware.

[0028] Perform topological sorting on the computation dependency graph, calculate the earliest start time and latest start time of each node, traverse the node set, find the nodes with the same earliest start time and latest start time as critical path nodes, perform topological sorting on the critical path nodes to obtain the critical path.

[0029] As one possible implementation, the subgraph segmentation and subgraph optimization of the standardized AI-Workflow IR intermediate representation workflow using the GRR algorithm includes:

[0030] A sliding window is used to extract all possible subgraphs in the critical path; for each possible subgraph, the node weights are calculated, which include the computational cost and model type of the node.

[0031] Traverse the predefined template subgraph set and extract the node weights and total computation for each template subgraph;

[0032] Select unmatched nodes in all possible subgraphs of the critical path and perform type matching with nodes in the template subgraph, and calculate weighted similarity. If the weighted similarity is greater than a set value, add it to the initial pairing set.

[0033] Verify that the nodes of the critical path subgraph in the initial pairing set are consistent with the edges of the nodes in the template subgraph; if they are, the matching is complete.

[0034] Design and initialize the target subgraph corresponding to the template subgraph in the pairing set;

[0035] Replace the target subgraphs corresponding to each subgraph in the initialization pairing set with the corresponding subgraphs in the critical path, and determine whether the replacement is truly performed based on the execution time before and after the replacement.

[0036] As one possible implementation, the agent matching and scheduling of the subgraph based on the deep reinforcement learning algorithm includes:

[0037] Define a state space, action space, and reward function. The state space consists of agent state, subgraph features, and higher-order features. Agent state includes CPU load rate, GPU load rate, used GPU memory, task queue length, and available network bandwidth. Subgraph features include computational cost of the subgraph to be processed, input data volume, required number of models, latency requirements, and whether it is a critical path subgraph. Higher-order features include capability matching degree, urgency score, total resource requirements, and load margin.

[0038] The action space is the action of assigning subgraphs to Agents;

[0039] The reward function is an immediate reward after the action is performed, including execution time reward, resource utilization reward and bandwidth reward;

[0040] The algorithm is trained using a deep reinforcement learning algorithm and outputs a matching and scheduling strategy for agents on subgraphs.

[0041] As one possible implementation, the method of training with a deep reinforcement learning algorithm and outputting an agent matching and scheduling strategy for the subgraph includes:

[0042] Training is performed using offline pre-training and online fine-tuning. In the offline pre-training stage, the model is trained based on data from the experience replay pool. The loss function includes policy network loss and value network loss. Training is completed when the average cumulative reward fluctuation over 50 consecutive rounds is less than or equal to 3% and the subgraph execution time achievement rate is greater than or equal to 80%. In the online fine-tuning stage, an ε-greedy strategy is used for fine-tuning.

[0043] Based on the trained model, state space data is collected to obtain the subgraph-agent allocation probability matrix, and the matching strategy with the highest probability is output.

[0044] As one possible implementation, it also includes generating a JIT-Agent to process the subgraph when the request frequency of a certain type of subgraph exceeds a frequency threshold, the execution time of the Agent of a certain type of subgraph is less than an execution latency threshold, or the GPU memory utilization of the Agent of a certain type of subgraph is greater than a utilization threshold.

[0045] As one possible implementation, the generation of the JIT-Agent includes:

[0046] The historical request subgraphs are clustered by features using a clustering algorithm, with each group corresponding to the service capabilities of a specialized agent.

[0047] Define the Agent based on the service capabilities of a specific type of Agent corresponding to each group;

[0048] Configure and deploy the defined Agent in the environment;

[0049] Register and update the metadata of the defined Agent.

[0050] On the other hand, the present invention provides a dynamic cloud scheduling system for artificial intelligence workflow, including a workflow intermediate representation transformation module, a critical path identification module, a subgraph segmentation optimization module, and an Agent matching module;

[0051] The workflow intermediate representation conversion module is used to define a standardized AI-Workflow IR intermediate representation and convert the native workflow into a workflow with a standardized AI-Workflow IR intermediate representation.

[0052] The critical path identification module is used to construct a computational dependency graph based on the workflow of the standardized AI-Workflow IR intermediate representation, obtain the critical path nodes in the computational dependency graph through the CPA algorithm, form the critical path, and mark the subgraphs in the critical path;

[0053] The subgraph segmentation and optimization module is used to perform subgraph segmentation and subgraph optimization on the standardized AI-Workflow IR intermediate representation of the workflow using the GRR algorithm.

[0054] The Agent matching module is used to match and schedule agents for subgraphs based on deep reinforcement learning algorithms.

[0055] As one possible implementation, the process of converting the native workflow into a standardized AI-Workflow IR intermediate representation includes:

[0056] Parse the original workflow, extract the node set, edge set, and attribute set of the original workflow, and extract the node type, input parameters, and metadata for each node;

[0057] Iterate through the input parameters of each node, identify static parameters, generate operators for static parameters, and add static constants to the constant set of the AI-Workflow IR module;

[0058] Based on the relationship between native nodes and IR operators of the AI-Workflow IR module, each node is mapped to an IR operator based on its node type;

[0059] Traverse the edge set of the native workflow, identify whether the input parameter of a node is a node reference. If so, use the output of the IR operator of the parent node of the node reference as the input of the IR operator of the lower node. If the input parameter of the node is identified as a static parameter, use the static constant corresponding to the node as the input of the IR operator.

[0060] Based on data dependencies, all operators are written into the @main function, metadata is added to each operator, and a complete standardized AI-Workflow IR intermediate representation is generated.

[0061] The beneficial effects of this invention are as follows: This invention introduces AI-Workflow IR, a unified intermediate representation, graph optimization algorithm, reinforcement learning dynamic scheduling, and dynamic microservice specialization, solving the pain points of existing technologies such as poor cross-framework compatibility, limited optimization methods, static scheduling, and weak microservice adaptability; AI-Workflow IR supports workflow conversion across different frameworks such as ComfyUI and Blender, eliminating the need to develop adaptation code for a single framework; it covers the entire workflow lifecycle from intermediate representation conversion, graph analysis, scheduling to execution, reducing execution time by 50%+; it optimizes scheduling strategies according to environmental changes, and JIT-Agent generates specialized services on demand, improving resource utilization by 25%+; client plugins enable seamless switching, eliminating the need for users to modify workflows or write cloud code, lowering the barrier to entry. Attached Figure Description

[0062] Figure 1 This is a flowchart of a dynamic cloud scheduling method for an artificial intelligence workflow.

[0063] Figure 2 This is a schematic diagram of a dynamic cloud scheduling system for an artificial intelligence workflow. Detailed Implementation

[0064] The present invention will be further described in detail below with reference to specific embodiments.

[0065] It should be noted that these embodiments are only used to illustrate the present invention and are not intended to limit the present invention. Simple improvements to the method under the premise of the present invention are all within the scope of protection claimed by the present invention.

[0066] Example 1

[0067] See Figure 1 This is a dynamic cloud scheduling method for artificial intelligence workflows, comprising:

[0068] S100. Define a standardized AI-Workflow IR intermediate representation to convert native workflows into workflows represented by the standardized AI-Workflow IR intermediate representation.

[0069] Advantages of standardized AI-Workflow IR intermediate representation:

[0070] It supports native workflow conversion across different frameworks (ComfyUI, Stable Diffusion WebUI), covering the core data types and operators of AI workflows.

[0071] Optimizability: The node computation characteristics, data dependencies, and model associations of the structured storage workflow provide metadata support for subsequent graph optimization (such as subgraph rewriting).

[0072] Scalability: Supports adding new data types (such as 3D mesh data !comfy.MESH) and operators (such as ControlNet inference comfy.ControlNetInfer) without refactoring the overall IR structure.

[0073] Lightweight: The IR code is small in size, which facilitates network transmission and memory loading, and avoids introducing additional performance overhead.

[0074] AI-Workflow IR defines two types of data: basic types and composite types, covering the full-scenario needs of AI workflows, as shown in Table 1.

[0075] Table 1. AI-Workflow IR Defines Data Types

[0076]

[0077] The operators in AI-Workflow IR correspond to the nodes of the native workflow. Each operator consists of four parts: operator name, input parameters, output type, and metadata. The core operator definitions are shown in Table 2.

[0078] Table 2 Definitions of core operators in AI-Workflow IR

[0079]

[0080] Each AI-Workflow IR corresponds to a complete AI workflow.

[0081] Workflows that convert native workflows into standardized AI-Workflow IR intermediate representations include:

[0082] The native workflow is a directed graph Graw = (Nraw, Eraw, Araw), where:

[0083] N raw = {n1, n2, ..., n m}: A collection of native nodes, each node being n i Includes classtype (node ​​type, such as KSampler), inputs (input parameters), and meta (metadata).

[0084] E raw = {(n i , n j ) | n i The output is n j Input}: The original edge set, representing data dependencies;

[0085] A raw = {a ij | a ij For n i The j-th attribute (e.g., seed, steps): the set of attributes.

[0086] The AI-Workflow IR module is defined as MIR = (CIR, FIR, OIR), where:

[0087] CIR: A set of constants (such as the parameters defined in the comfy.constant operator);

[0088] FIR: Function set (containing only the @main function, corresponding to workflow execution logic);

[0089] OIR: A set of operators (such as comfy.LoadImage, comfy.KSampler).

[0090] The transformation process includes:

[0091] (1) Parse the original workflow, extract the node set, edge set, and attribute set of the original workflow, and for each node n i Extract node type class type(e.g., "KSampler"), input parameters (input parameter name and value), meta (nodeid, title).

[0092] (2) Traverse each node n i input parameters i Identify static parameters (e.g., seed=280823642470253, ckpt_name="dreamshaper_8.safetensors"); for each static parameter (k ij ,v ij ), generate the comfy.constant operator, convert it into a static constant and add it to the constant set CIR; dynamic parameters (such as parameters that depend on the output of other nodes) are not processed for the time being, and are left for step (4).

[0093] (3) Based on the original node-IR operator mapping relationship, n i .class type Mapped to IR operator o i (e.g., "KSampler" → comfy.KSampler); for each operator o i Extract its input parameter type list T in With output type list T out (such as comfy.KSampler's T) in = [!comfy.INT, !comfy.INT, ...]); Verify n i The number of input parameters and T in Check if the length matches; if not, report an error (type incompatibility).

[0094] (4) Traverse edge set E raw , identify n i Input parameter v ij Whether it is a node reference (e.g., ["14", 0] represents the 0th output of node 14); if v ij For node references (n k i dx If n is found, then n is found. k The corresponding operator o k The i-th dx Output out kidx , will out kidx As o i The input; if v ij If it is a static parameter, then find the constant c generated in step (2). ij , will c ij As o i Input.

[0095] (5) Write all operators into the @main function in order of data dependency (topological sorting); add metadata (node) for each operator. id , title, param names ); Generate a complete standardized AI-Workflow IR intermediate representation: MIR = (CIR, FIR, OIR), where FIR = {@main}, OIR = {o i}

[0096] Time complexity: Let the number of nodes in the original workflow be m and the number of edges be e. Then the time complexity of step (1) is O(m + e); step (2) is O(m × k) (k is the average number of parameters of the nodes); step (3) (node-operator mapping) is O(m) (mapping table lookup is O(1)); step (4) (edge-data dependency mapping) is O(e); step (5) (IR assembly) is O(m + e) ​​(topology sorting is O(m + e)). The total time complexity is O(m × k + m + e) ​​= O(m × k) (since k is usually a constant, it can be simplified to O(m)), which meets the real-time conversion requirements of large-scale workflows (such as m=1000).

[0097] Space complexity: The space complexity of the IR module is O(m × (k + 1)) (each node corresponds to 1 operator + k constants). Since k is a constant, it can be simplified to O(m) to avoid memory overflow.

[0098] Validation of AI-Workflow IR intermediate representation:

[0099] To ensure that the converted IR intermediate representation can be correctly processed by the subsequent optimization and scheduling modules, the syntax correctness and type consistency must be verified through AI-Workflow IR intermediate representation verification. The verification rules are shown in Table 3.

[0100] Table 3. AI-Workflow IR Intermediate Representation Validation Rules

[0101]

[0102] S200, based on the standardized AI-Workflow IR intermediate representation of the workflow, constructs a computational dependency graph, and uses the CPA algorithm to obtain the critical path nodes in the computational dependency graph, thus forming the critical path.

[0103] Based on the @main function of the standardized AI-Workflow IR intermediate representation, a computation dependency graph G is constructed. dep = (V dep E dep Wdep ),in:

[0104] V dep = {v1, v2, ..., v m}: A set of nodes, each node v i One of the operators o in IR i ;

[0105] E dep = {(v i , v j ) | o i The output is o j Input}: Edge set, representing the data dependencies between operators;

[0106] W dep = {w i | w i For operator o i Estimated execution time: weight set, w i o i Execution time on standard hardware (such as the A100 GPU) (in milliseconds).

[0107] The CPA algorithm is based on topological sorting, calculating the earliest start time (EST) and latest start time (LST) of each node, with the critical path being the sequence of nodes where "EST = LST". The specific process includes:

[0108] (1) Topological sorting: for computational dependency graph G dep Perform a topological sort to obtain the node order ord = [v1, v2, ..., v m (Ensure that all predecessor nodes precede the current node).

[0109] (2) Calculate the earliest start time (EST): The EST of the starting node (a node without a predecessor, such as comfy.LoadImage) is 0; for other nodes v j EST(v j = max(EST(v1) + w1) (v1 is v) j (All predecessor nodes).

[0110] (3) Calculate the latest start time (LST): The LST of the terminating node (a node with no successor, such as comfy.SaveImage) = EST(v end (Total execution time); For other nodes v1, LST(v1) = min(LST(v1) / v2) = min(LST(v1) / v2) / v3) j ) - w1) (v j(All successor nodes of v1).

[0111] (4) Identify the critical path: Go through all nodes. If EST(v1) = LST(v1), then v1 is a critical path node. The critical path is a sequence of critical path nodes arranged in topological order.

[0112] Prioritize optimization of the critical path: When splitting and scheduling subgraphs in subsequent subgraphs, prioritize allocating subgraphs on the critical path to high-performance agents (such as A100 GPUs) to shorten the total execution time;

[0113] Parallel potential identification: Nodes on non-critical paths (EST < LST) have slack time and can be executed in parallel with critical path nodes, improving resource utilization.

[0114] S300. The GRR algorithm is used to perform subgraph segmentation and subgraph optimization on the workflow of the standardized AI-Workflow IR intermediate representation.

[0115] Subgraph feature extraction: A sliding window is used to extract all possible subgraphs G in the critical path. candidate (Window size is 2~10 nodes); for each G candidate The node weight w(v) is calculated as α×comp(v) + β×model(v) (comp(v) is the computational cost of node v, model(v) is the model type encoding, and α+β=1).

[0116] Template traversal: Traverse the predefined template sub-graphet T, for each template T i Extract its node weight w T (v T ) and total computational cost Comp(T) i ).

[0117] Select all possible unmatched nodes v ∈ G in the critical path's subgraph. candidate With node v in the template subgraph T ∈ T i Perform matching; if the operator types are the same and the weight similarity is sim(w(v), w T (v T If )) ≥ θ, then add it to the initial pairing set M;

[0118] Verify whether M satisfies edge consistency ((v1,v2)∈E(G) candidate ) ↔ (M(v1), M(v2))∈E(T i If the conditions are met, the matching is complete;

[0119] Design and initialize the target subgraph corresponding to the template subgraph in the pairing set, ensuring that the target subgraph T i The input nodes of ' are consistent with the input nodes of the template subgraph in terms of type and number, and the output nodes are consistent with the output nodes of the template subgraph in terms of type and number.

[0120] In the critical path, the target subgraphs corresponding to each subgraph in the initialization pairing set are replaced with the corresponding subgraphs in the critical path, i.e., M is deleted. match Insert T with all nodes and edges of the corresponding critical path. i ' ' nodes and edges, and establish InT i 'With In match Dependencies, OutT i 'and Out match Dependency relationships.

[0121] Execute the replacement subgraph T i ', Compare the execution time Comp(M) before and after the replacement match ) and Comp(T i If Comp(T) i ') ≥ Comp(M match If the value is not specified, roll back and replace it (to avoid a decrease in efficiency).

[0122] Execution time can be estimated dynamically by combining operator type, input data volume, and hardware type, and calculated using a linear regression model.

[0123] Feature extraction: for operator o i Extracting feature vector X i = [x1, x2, x3, x4], where:

[0124] x1: Operator type encoding (e.g., comfy.KSampler=1, comfy.CLIPTextEncode=2);

[0125] x2: Input data volume (e.g., the number of pixels in comfy.IMAGE H×W, the number of elements in comfy.LATENT N×C×H×W).

[0126] x3: Hardware performance coefficient (e.g., A100 GPU=1.0, RTX 3090=0.6, CPU=0.1).

[0127] x4: Model parameter size (e.g., the number of parameters P in comfy.MODEL, in B).

[0128] Regression model: The regression coefficients β = [β0, β1, β2, β3, β4] are obtained through offline training (collecting execution time samples of different operators and hardware). Then w i = β0 + β1x1 + β2x2+ β3x3 + β4x4;

[0129] Online correction: Each time the operator is executed, the actual execution time w is adjusted. i ^actual and estimated time w i The error of ^pred is fed back to the model to update the regression coefficient β and improve the accuracy of subsequent predictions.

[0130] Examples of target subgraphs corresponding to template subgraphs in the initialization pairing set are shown in Table 4.

[0131] Table 4. Example table of target subgraphs corresponding to template subgraphs

[0132]

[0133] Time complexity: The time complexity is O((m + n) × 2^d), where m is the number of nodes in the original subgraph, n is the number of nodes in the template subgraph, and d is the depth (the number of backtracking steps in the matching process). Since the subgraph window size of the AI ​​workflow is ≤10 (m≤10), the number of nodes in the template subgraph is ≤10 (n≤10), and d≤5, the time complexity can be simplified to O(1), which meets the requirements of real-time optimization.

[0134] Space complexity: It mainly stores the template subgraph set T and the target subgraph set T'. The space complexity of each template is O(n), and the total space complexity of T is O(k×n) (k is the number of templates, usually k≤50), which can be ignored.

[0135] Cross-node redundancy elimination based on dynamic programming:

[0136] In existing technologies, cross-node data transmission (such as Agent A outputting data to the cloud and Agent B downloading it) involves significant redundancy (e.g., multiple agents repeatedly downloading the same data), leading to wasted network bandwidth. This invention proposes a CRE (Cross-node Redundancy Elimination) algorithm, which identifies and eliminates cross-node redundancy based on dynamic programming.

[0137] (1) Redundancy type definition

[0138] Cross-node redundancy in AI workflows can be mainly divided into two categories:

[0139] Data transmission redundancy: Multiple subgraphs S1, S2, ..., S kThe same data D (such as the comfy.CLIP model and the comfy.IMAGE image) is required, and D needs to be transferred from the local machine or other Agents to the Agents where each subgraph is located, resulting in duplicate transmission;

[0140] Computational redundancy: Multiple subgraphs S1, S2, ..., S k Performing the same calculation (such as comfy.VAEEncode on the same image) yields the same result, leading to duplicate calculations.

[0141] (2) The core idea of ​​the CRE algorithm

[0142] For data transmission redundancy: Store data D in the Agent closest to all demand subgraphs (such as a shared storage Agent), and each subgraph reads D from this Agent to avoid duplicate transmission; For computation redundancy: Identify subgraphs with the same computation, perform the computation only once, and share the result with other subgraphs to avoid duplicate computation.

[0143] (3) Steps of the CRE algorithm (taking data transmission redundancy as an example)

[0144] The CRE algorithm uses dynamic programming to solve for the minimum redundancy cost. The specific steps are as follows:

[0145] 1) Data demand and transmission cost modeling:

[0146] Traverse the optimized, standardized AI-Workflow IR intermediate representation (after subgraph partitioning and optimization) to identify all data transmitted across agents, D = {d1, d2, ..., d...} p};

[0147] For each data d j Identify the set of demand subgraphs S j = {s1, s2, ..., s k}(s i d is required j );

[0148] For each d j Calculate the data transmission cost matrix C j ∈ R^{(m+1)×k}, where m is the number of Agents, C j[t][i] For the transmission of d from source t j To subgraph s i Cost of the Agent (t=0 represents local, t=1..m represents Agent 1..m); Transmission cost C j[t][i] = (Size(d j) / Bandwidth(t,i)) + Latency(t,i)(Size(d j (t,i) represents the data size, Bandwidth(t,i) represents the bandwidth from source t to Agent i, and Latency(t,i) represents the latency.

[0149] 2) Dynamic Programming State Definition

[0150] Define state d p[j][t] : for data d j When source t (t=0..m) is selected, all demand subgraphs s i ∈ S j Total transmission cost.

[0151] 3) Optimal source selection

[0152] For each data d j Choose to make d p[j][t] Minimum source top t top t = argmin{t∈0..m} dp [j][t] .

[0153] 4) Redundancy elimination execution

[0154] Data d j From the source top t Transfer to shared storage point (if top) t If it is an Agent, then the shared storage point is that Agent; if top t If it is local, then d j Upload to the Agent with the lowest load as a shared storage point);

[0155] All demand subgraphs i ∈ S j Read from shared storage point d j This replaces the original method of transmitting data from their respective sources.

[0156] 5) Calculate redundancy elimination

[0157] For all subgraphs, calculate the feature hash h(s) = hash(class_type(s) + input_hash(s)), where input_hash(s) is the hash value of all input data of subgraph s;

[0158] If there are multiple subgraphs s1, s2, ..., s k h(s) i If the results are the same, then only s1 is executed, and the result Res(s1) is shared with s2...s k .

[0159] S400: Agent matching and scheduling for subgraphs based on deep reinforcement learning algorithms.

[0160] Environment definition:

[0161] State space S: states s t ∈ S represents the environment state at time t, which consists of the Agent state and subgraph features:

[0162] Agent status S agent = [s a1 , s a2 , ..., s am ], where m is the number of Agents, and each S ai =[load cpu load gpu mem used model cache task queue band width ](load cpu CPU load rate, load gpu For GPU load rate, mem used For the used GPU memory, model cache For model cache matching degree, task queue Band is the length of the task queue. width (Including available network bandwidth), you can also add available software interfaces to divide the capabilities of the Agent;

[0163] Subgraph features Ss ub = [s1, s2, ..., s q ], where q is the number of subgraphs to be scheduled, and each s j = [comp j ,data j model j latency req critica j ](comp j For subgraph computation, data j For the amount of input data, model j For the required number of models, latency req To delay demand, critical j (Whether it is a critical path subgraph).

[0164] Higher-order features are used to characterize the relationship between the subgraph and the agent, including: capability matching degree, urgency score, total resource requirement, and load margin.

[0165] Ability matching degree: , It is the matching ratio between the model required by the subgraph and the model cached by the Agent (e.g., if the subgraph requires 3 models and the Agent caches 2, then it is 2 / 3). It is a normalized value of the GPU memory used by the Agent, which measures the Agent's hardware and model support for the current subgraph. The higher the value, the better the matching degree and the higher the scheduling priority.

[0166] Urgency score: , It is a binary identifier for whether a subgraph is a critical path (1 for critical paths, 0 otherwise). It is the normalized value of the delayed demand of the subgraph. It is the maximum value of the delay requirements of all subgraphs, reflecting the urgency of the subgraph's execution. Subgraphs with critical paths and strict delay requirements have high scores and should be scheduled first.

[0167] Total resource requirements: , It is the normalized value of the subgraph computation. It is a normalized value of the input data volume of the subgraph. Considering the computation and data transmission requirements of the subgraph, a higher value indicates greater resource consumption, and a higher-specification Agent needs to be matched during scheduling.

[0168] Load margin: , This is the normalized value of the Agent CPU load. This is a normalized value of the Agent's GPU load, which measures the Agent's remaining resource capacity. A higher value indicates that the CPU and GPU are more idle and can handle more subgraph tasks.

[0169] Suppose there are m agents in the cloud and q subgraphs to be scheduled, then the final state vector... The dimension is calculated as follows:

[0170] The feature dimensions of a single Agent are: 6 (Agent state) + 0 (Agent itself has no higher-order features) = 6.

[0171] The feature dimension of a single subgraph is: 5 (subgraph features) + 4 (higher-order features) = 9.

[0172] Global state dimension: m×6 + q×9.

[0173] Action space A: Action at ∈ A is the scheduling decision at time t.

[0174] Level 1 (Subgraph Priority Sort): Sort the q subgraphs according to their urgency scores to obtain the execution order. (Prioritize scheduling of emergency subgraphs);

[0175] Second layer (Agent assignment): For each sorted subgraph s j Choose one Agent from m Agents for assignment, and represent the action as a. t = [a1, a2, ..., a q ], where a j ∈{1, 2, ..., m}(a j =i represents the subgraph s j Assigned to Agent i ).

[0176] To prevent the policy network from generating invalid actions (such as assigning a GPU-required subgraph to a CPU agent), pre-filtering is performed:

[0177] Hardware capability constraints: If the subgraph contains GPU operators (such as KSampler, VAEEncode), it can only be assigned to agents with GPUs;

[0178] Memory constraint: The Agent's remaining video memory must be greater than or equal to the video memory required by the subgraph;

[0179] Model constraints: If the model required for the subgraph is not cached in the Agent and cannot be downloaded in real time (e.g., model size > 10GB and bandwidth < 1Gbps), then the Agent is excluded;

[0180] Load constraint: The current load rate of the Agent must be < 0.9 (to avoid overload).

[0181] After filtering, the set of optional Agents A for each subgraph j ∈{1, ..., m}, the policy network is only in A j Selecting actions within the framework significantly reduces ineffective exploration.

[0182] Encoding: Action a t = [a1, ..., a q The result is converted to one-hot encoding with a total dimension of q×m, which serves as the input to the value network.

[0183] Decoding: The policy network outputs a probability matrix through the Softmax activation function. , This represents the probability that subgraph j is assigned to Agent i, for each subgraph s j In A jChoose the Agent with the highest probability In actual scheduling, the action with the highest probability is prioritized (using the current optimal strategy); to avoid getting trapped in local optima, an ε-greedy exploration mechanism is introduced, based on probability. Randomly select an action (explore new solutions) with a probability of 1- Choose the optimal action.

[0184] Reward function R: Reward r t = R(s t , a t ) to perform action a t The immediate reward afterward needs to balance three objectives: execution time, resource utilization, and network bandwidth.

[0185] Execution time reward r time :

[0186] ;

[0187] in, This represents the actual execution time of the subgraph. For delayed requirements of a subgraph, when completed on time, the reward increases as the actual time decreases; when overdue, a negative reward is given, and the penalty intensity increases with the overdue percentage.

[0188] Resource utilization rate reward r util :

[0189] ;

[0190] Network overhead reward r bw To measure the efficiency of agent resource utilization:

[0191] ;

[0192] in, The amount of data transmitted for the subgraph. The lower the proportion of the input volume to the bandwidth-time product, the higher the reward.

[0193] Total Rewards: ; This is a penalty item.

[0194] Training deep reinforcement learning algorithms:

[0195] Train two value networks simultaneously and (The parameters are respectively) , Both have the same structure but independent parameters. When calculating the value of a state-action pair, the minimum of the two values ​​is taken. This avoids overestimation of a single network.

[0196] To further stabilize training, a target value network with the same structure as the main network was set up. and Its update frequency is lower than that of the main network: every N=100 steps (after the main network updates 100 times), the target network parameters are synchronized with the main network parameters through soft update rules. ,in The coefficients are soft-update coefficients to ensure that the target network parameters change slowly and avoid drastic fluctuations in value estimation.

[0197] The training adopts a two-stage mode of offline pre-training and online fine-tuning, which ensures the effectiveness of the initial strategy and can adapt to the dynamic changes in the real environment.

[0198] (1) Offline pre-training stage:

[0199] 1) Data collection and experience replay pool construction:

[0200] Simulation scenario design: Collect or simulate 10,000 typical scenarios, including:

[0201] Agent status changes: GPU load 20%~90%, VRAM usage 10%~80%, model cache count 0~5;

[0202] Subgraph feature variations: computational cost 50~1000 GFLOPs, data size 10~500 MB, latency requirement 500~5000 ms;

[0203] Interference factors: random agent failure (probability 5%), network bandwidth fluctuations (1~10 Gbps).

[0204] Trajectory generation: For each scene, a random strategy is used to generate and assign actions (uniformly random selection of a). t Record the complete trajectory after execution. (Status-Action-Reward-Next Status), cumulative build capacity is 10 6 The experience replay pool R.

[0205] 2) Loss function and parameter update:

[0206] Policy network loss ( The goal of the policy network is to maximize the value of state-action pairs, and the loss function is defined as the negative expected value (maximizing value through gradient ascent).

[0207] ;

[0208] in It is the action output by the policy network. It is the value network's estimate of the value of this action.

[0209] Value network loss ( The goal of a value network is to minimize the difference between the predicted value and the target value, where the target value includes both immediate rewards and future value.

[0210] ;

[0211] Among them, the target value ( =0.9 is the discount factor. It is the next action output by the target policy network.

[0212] 3) Training parameters:

[0213] Optimizer: AdamW is used to suppress overfitting;

[0214] Learning rate: Policy network Value network ;

[0215] Batch size (B=256), training rounds E=500 (B trajectories are randomly sampled from the experience replay pool in each iteration);

[0216] Update order: Update the policy network once every 5 iterations of the value network (optimize the policy only after the value estimation is stable).

[0217] When the average cumulative reward fluctuation over 50 consecutive rounds is ≤3%, and the subgraph execution time meets the target rate When the accuracy is ≥80%, stop pre-training and save the network parameters as the initial values ​​for online fine-tuning.

[0218] (2) Online fine-tuning stage: Continuously optimize the strategy in a real cloud environment to solve the distribution offset problem between the simulated environment and the real environment.

[0219] 1) Exploring and utilizing equilibrium (ε-greedy strategy):

[0220] Initial exploration rate =0.5 (50% probability of trying a new move), decays exponentially with the number of training steps t: ,

[0221] When t > 10 5 (At 100,000 steps) The rate is kept stable at 0.01 (1% exploration rate), with utilization as the primary focus and exploration as a secondary approach.

[0222] Special scenario adjustment: If a new type of subgraph is detected (feature difference from history > 3σ), temporarily adjust... Increase to 0.3 and continue for 200 steps to quickly adapt to new tasks.

[0223] 2) Environmental feedback and parameter updates:

[0224] Feedback trigger: A feedback is triggered once the entire process of a subgraph is completed (from assignment to result return).

[0225] Track recording: recording in the real environment Add to the experience replay pool R (while discarding the oldest 10% of old data to maintain the timeliness of the pool).

[0226] Parameter update: Each feedback sample B=64 trajectories from R (small batch for accelerated real-time updates), and the network parameters are updated according to the loss function of the offline stage. The target network is still soft-updated every 100 steps.

[0227] The process of matching and scheduling agents on a subgraph is as follows:

[0228] State Acquisition: The Agent State Monitoring Module collects the state of all Agents in real time, and the Subgraph Feature Module extracts the subgraph features of the subgraph to be scheduled and combines them into a state vector s. t ;

[0229] Action generation: s t Input the trained policy network, output the subgraph - Agent assignment probability matrix P, and select the assignment scheme a with the highest probability. t (like < Exploration Rate If a random allocation scheme is selected, it will be explored.

[0230] Perform multi-objective decision verification a t Check if the constraints are met (e.g., whether the Agent's video memory is sufficient to load the model required by the subgraph, and whether the bandwidth meets the data transmission requirements). If not, regenerate the action (select the suboptimal probability scheme).

[0231] After serializing the subgraph (based on the binary encoding of AI-Workflow IR), it is sent to the corresponding Agent and the scheduling log (subgraph ID, Agent ID, allocation time) is recorded.

[0232] After the subgraph execution is completed, the reward is calculated based on the actual execution time, Agent load changes, and bandwidth usage. ,Will Add to the experience replay pool to trigger online fine-tuning.

[0233] Cloud-based microservices (Agents) have fixed capabilities (such as pre-deployed general AI inference agents and CLIP-encoded agents), making it impossible to adjust service capabilities based on the dynamic characteristics of user requests (such as a surge in requests for a certain type of subgraph or the emergence of new types of subgraphs). This results in low resource utilization and high response latency. This invention proposes a dynamic microservice specialization mechanism (JIT-Agent mechanism) based on request clustering and resource prediction, aiming to achieve the goal of generating specialized agents on demand and automatically evolving service capabilities.

[0234] General Agent: A pre-deployed fallback agent that supports all / many types of AI workflow subgraphs (based on a complete model library and node library), but has low execution efficiency (frequent model loading and high memory usage).

[0235] Specialized Agent: An agent dynamically generated based on the JIT mechanism, which only supports one or a few types of subgraphs with similar features. It achieves efficient execution through model preloading, operator optimization, and data caching.

[0236] The generation of a specialized agent is triggered when any of the following conditions are met:

[0237] Request frequency threshold for a certain type of subgraph: Within a continuous T=5 minutes, the number of requests for the same feature subgraph is ≥K=20 (the feature is determined by the feature hash of the subgraph).

[0238] General Agent execution latency threshold: The average execution time of a certain type of subgraph on the general agent is ≥ T{max}=2000ms (exceeding the estimated execution time of the specialized agent T_{exp}=500ms);

[0239] Resource utilization threshold: GPU memory utilization of the general agent ≥ U_{max}=85% (services need to be split to avoid OOM).

[0240] The generation of JIT-Agent includes:

[0241] (1) Group the subgraphs of historical requests by features using a clustering algorithm. Each group corresponds to the service capabilities of a specialized agent. The steps are as follows:

[0242] Feature extraction: For each historical subgraph s, extract the feature vector. :

[0243] Subgraph computation cost (unit: GFLOPs, obtained by summing operator computation costs, e.g., KSampler operator computation cost = number of steps × latent size × number of model parameters).

[0244] Required model type encoding (e.g., SD 3.5=1, FLUX.1=2, CLIP=3, VAE=4).

[0245] Input data type encoding (e.g., IMAGE=1, LATENT=2, CONDITIONING=3);

[0246] : Number of nodes in the subgraph (number of IR operators contained in the subgraph);

[0247] Average execution time (the average of historical execution times, in milliseconds).

[0248] Feature standardization: for Z-score standardization is performed to eliminate the influence of dimensions.

[0249] K-Means++ clustering:

[0250] Initialize cluster centers: Use the K-Means++ sampling strategy. The first center is randomly selected, and subsequent centers are selected from the samples that are farthest from the existing centers.

[0251] Iterative clustering: Calculate the Euclidean distance from each sample in the subgraph to each center. The sample is assigned to the nearest cluster C. k ;

[0252] Update center: For each cluster C k Computing Center ;

[0253] Termination condition: Stop iteration when the change in cluster centers is ≤ ϵ = 1e−4 or the number of iterations is ≥ I = 100;

[0254] Clustering validity verification: Calculating the silhouette coefficient to assess clustering quality.

[0255] ;

[0256] in, Let s be the average distance between s and other samples within the same cluster. SC represents the average distance between s and the nearest heterogeneous cluster sample; the closer SC is to 1, the better the clustering effect.

[0257] (2) For each valid cluster C k Define the capabilities of the corresponding specialized Agent:

[0258] Model preloading list: Statistics C k The models required for all subgraphs are deduplicated to form a preload list. (e.g. C)k If it is "SD 3.5 Diffusion Inference Subgraph", then );

[0259] Operator optimization configuration: for C k For frequently occurring operators (such as KSampler and VAEDecode), enable hardware acceleration optimizations (such as TensorRT quantization and CUDA kernel optimization).

[0260] Data caching strategy: for C k For high-frequency input data of the inner subgraph (such as CLIP encoding results of fixed Prompt), set a cache expiration period to avoid redundant calculations;

[0261] Resource demand forecasting: based on C k Maximum memory usage of inner subgraph Estimated memory requirements for specialized agents (Reserving 20% ​​redundancy), CPU core requirements (Assume a single CPU core has a computing power of 100 GFLOPs).

[0262] (3) Agent environment configuration and deployment

[0263] Based on the Kubernetes container orchestration platform, the deployment of specialized agents is completed automatically:

[0264] Container image generation: Based on the basic AI inference image (including CUDA, PyTorch, and MLIR runtime), integrating... The model, optimization operator library, and data caching module are used to generate a lightweight image (typically 60% to 80% smaller than a general Agent image).

[0265] Resource Request: Submit Pod configuration to the Kubernetes API to request resources. One core, Video memory, bandwidth( Based on C k (Calculation of average data volume and request frequency of inner subgraphs).

[0266] Service registration: After a Pod starts, it is automatically registered with a service discovery component (such as Consul) and exposes a gRPC interface (used to receive subgraph execution requests).

[0267] Health checks: Kubernetes performs health checks periodically (such as sending test subgraph requests). If the Agent fails to perform three consecutive checks, the Pod will be automatically restarted or rebuilt.

[0268] (4) Meta-information registration and update: The metadata (capabilities, resource status, interface address) of the specialized agent is registered to the CGR scheduling layer in real time. The update process is as follows:

[0269] When the Agent starts, it sends a registration request to the CGR scheduling layer, carrying metadata. :

[0270] Specialized Agent Unique Identifier (e.g., “SD3.5-Infer-Agent-001”);

[0271] Capability description (supported subgraph feature clustering (C_k), model list (M_k), operator list);

[0272] Current resource status (CPU / GPU load, video memory usage, bandwidth);

[0273] : gRPC interface address (e.g., "10.0.0.1:50051");

[0274] The CGR scheduling layer updates the Agent metadata database, and in subsequent scheduling, it prioritizes C... k Subgraph assigned to The corresponding Agent;

[0275] During agent operation, a metadata update request is sent to the CGR scheduling layer every 10 seconds. If the resource status exceeds the threshold (e.g., GPU load ≥ 90%), it is marked as busy, and the scheduling layer will not allocate new tasks for the time being.

[0276] See Figure 2 The above-mentioned method forms a dynamic cloud scheduling system for artificial intelligence workflow, including a workflow intermediate representation conversion module 100, a critical path identification module 200, a subgraph segmentation optimization module 300, and an agent matching module 400.

[0277] The workflow intermediate representation conversion module 100 is used to define a standardized AI-Workflow IR intermediate representation and convert the native workflow into a workflow with a standardized AI-Workflow IR intermediate representation.

[0278] The critical path identification module 200 is used to construct a computational dependency graph based on the standardized AI-Workflow IR intermediate representation of the workflow, obtain the critical path nodes in the computational dependency graph through the CPA algorithm, form the critical path, and mark the subgraphs in the critical path;

[0279] The subgraph segmentation and optimization module 300 is used to perform subgraph segmentation and subgraph optimization on the standardized AI-Workflow IR intermediate representation of the workflow using the GRR algorithm;

[0280] Agent matching module 400 is used to match and schedule agents for subgraphs based on deep reinforcement learning algorithms.

[0281] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described with reference to preferred embodiments, those skilled in the art should understand that various changes in form and detail can be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A dynamic cloud scheduling method for artificial intelligence workflows, characterized in that, include: Define a standardized AI-Workflow IR intermediate representation to convert native workflows into workflows represented by the standardized AI-Workflow IR intermediate representation; define two types of data types for AI-Workflow IR: basic types and composite types; the operators of AI-Workflow IR correspond to the nodes of the native workflow, and each operator contains four parts: operator name, input parameters, output type, and metadata. The core operators include data recording, model loading, text encoding, diffusion inference, image decoding, and image saving; each AI-Workflow IR corresponds to a complete AI workflow. Based on the standardized AI-Workflow IR intermediate representation of the workflow, a computational dependency graph is constructed. The critical path nodes in the computational dependency graph are obtained through the CPA algorithm, forming the critical path. The subgraphs in the critical path are then marked. The GRR algorithm is used to perform subgraph segmentation and subgraph optimization on the standardized AI-Workflow IR intermediate representation of the workflow; Agent matching and scheduling are performed on subgraphs based on deep reinforcement learning algorithms.

2. The dynamic cloud scheduling method for artificial intelligence workflows according to claim 1, characterized in that, The process of converting the native workflow into a standardized AI-Workflow IR intermediate representation includes: Parse the original workflow, extract the node set, edge set, and attribute set of the original workflow, and extract the node type, input parameters, and metadata for each node; Iterate through the input parameters of each node, identify static parameters, generate operators for static parameters, and add static constants to the constant set of the AI-Workflow IR module; Based on the relationship between native nodes and IR operators of the AI-Workflow IR module, each node is mapped to an IR operator based on its node type; Traverse the edge set of the native workflow, identify whether the input parameter of a node is a node reference. If so, use the output of the IR operator of the parent node of the node reference as the input of the IR operator of the lower node. If the input parameter of the node is identified as a static parameter, use the static constant corresponding to the node as the input of the IR operator. Based on data dependencies, all operators are written into the @main function, metadata is added to each operator, and a complete standardized AI-Workflow IR intermediate representation is generated.

3. The dynamic cloud scheduling method for artificial intelligence workflow according to claim 2, characterized in that, The construction of the computation dependency graph involves using the CPA algorithm to derive the critical path nodes within the graph, forming the critical paths, including: Based on the @main function of the standardized AI-Workflow IR intermediate representation, a computational dependency graph containing a set of nodes, a set of edges, and a set of weights is constructed. The nodes in the node set represent an operator, the edges in the edge set represent the data dependencies between operators, and the weights in the weight set represent the execution time of the operator on standard hardware. Perform topological sorting on the computation dependency graph, calculate the earliest start time and latest start time of each node, traverse the node set, find the nodes with the same earliest start time and latest start time as critical path nodes, perform topological sorting on the critical path nodes to obtain the critical path.

4. The dynamic cloud scheduling method for artificial intelligence workflows according to claim 3, characterized in that, The subgraph segmentation and optimization of the standardized AI-Workflow IR intermediate representation using the GRR algorithm includes: A sliding window is used to extract all possible subgraphs in the critical path; for each possible subgraph, the node weights are calculated, which include the computational cost and model type of the node. Traverse the predefined template subgraph set and extract the node weights and total computation for each template subgraph; Select unmatched nodes in all possible subgraphs of the critical path and perform type matching with nodes in the template subgraph, and calculate weighted similarity. If the weighted similarity is greater than a set value, add it to the initial pairing set. Verify that the nodes of the critical path subgraph in the initial pairing set are consistent with the edges of the nodes in the template subgraph; if they are, the matching is complete. Design and initialize the target subgraph corresponding to the template subgraph in the pairing set; Replace the target subgraphs corresponding to each subgraph in the initialization pairing set with the corresponding subgraphs in the critical path, and determine whether the replacement is truly performed based on the execution time before and after the replacement.

5. The dynamic cloud scheduling method for artificial intelligence workflows according to claim 4, characterized in that, The agent matching and scheduling of the subgraph based on the deep reinforcement learning algorithm includes: Define a state space, action space, and reward function. The state space consists of agent state, subgraph features, and higher-order features. Agent state includes CPU load rate, GPU load rate, used GPU memory, task queue length, and available network bandwidth. Subgraph features include computational cost of the subgraph to be processed, input data volume, required number of models, latency requirements, and whether it is a critical path subgraph. Higher-order features include capability matching degree, urgency score, total resource requirements, and load margin. The action space is the action of assigning subgraphs to Agents; The reward function is an immediate reward after the action is performed, including execution time reward, resource utilization reward and bandwidth reward; The algorithm is trained using a deep reinforcement learning algorithm and outputs a matching and scheduling strategy for agents on subgraphs.

6. The dynamic cloud scheduling method for artificial intelligence workflow according to claim 5, characterized in that, The method of training with a deep reinforcement learning algorithm and outputting an agent matching and scheduling strategy for the subgraph includes: Training is performed using offline pre-training and online fine-tuning. In the offline pre-training stage, the model is trained based on data from the experience replay pool. The loss function includes policy network loss and value network loss. Training is completed when the average cumulative reward fluctuation over 50 consecutive rounds is less than or equal to 3% and the subgraph execution time achievement rate is greater than or equal to 80%. In the online fine-tuning stage, an ε-greedy strategy is used for fine-tuning. Based on the trained model, state space data is collected to obtain the subgraph-agent allocation probability matrix, and the matching strategy with the highest probability is output.

7. The dynamic cloud scheduling method for artificial intelligence workflow according to claim 6, characterized in that, It also includes generating a JIT-Agent to process a certain type of subgraph when the request frequency of a certain type of subgraph exceeds a frequency threshold, the execution time of an Agent for a certain type of subgraph is less than an execution latency threshold, or the GPU memory utilization of an Agent for a certain type of subgraph is greater than a utilization threshold.

8. The dynamic cloud scheduling method for artificial intelligence workflow according to claim 7, characterized in that, The generation of the JIT-Agent includes: The historical request subgraphs are clustered by features using a clustering algorithm, with each group corresponding to the service capabilities of a specialized agent. Define the Agent based on the service capabilities of a specialized Agent corresponding to each group; Configure and deploy the defined Agent in the environment; Register and update the metadata of the defined Agent.

9. A dynamic cloud-based scheduling system for artificial intelligence workflows, characterized in that, It includes a workflow intermediate representation transformation module, a critical path identification module, a subgraph segmentation optimization module, and an Agent matching module; The workflow intermediate representation conversion module is used to define a standardized AI-Workflow IR intermediate representation, converting the native workflow into a workflow represented by the standardized AI-Workflow IR intermediate representation. AI-Workflow IR defines two types of data: basic types and composite types. The operators in AI-Workflow IR correspond to the nodes of the native workflow. Each operator includes four parts: operator name, input parameters, output type, and metadata. Core operators include data recording, model loading, text encoding, diffusion inference, image decoding, and image saving. Each AI-Workflow IR corresponds to a complete AI workflow. The critical path identification module is used to construct a computational dependency graph based on the workflow of the standardized AI-Workflow IR intermediate representation, obtain the critical path nodes in the computational dependency graph through the CPA algorithm, form the critical path, and mark the subgraphs in the critical path; The subgraph segmentation and optimization module is used to perform subgraph segmentation and subgraph optimization on the standardized AI-Workflow IR intermediate representation of the workflow using the GRR algorithm. The Agent matching module is used to match and schedule agents for subgraphs based on deep reinforcement learning algorithms.

10. The dynamic cloud scheduling system for artificial intelligence workflows according to claim 9, characterized in that, The process of converting the native workflow into a standardized AI-Workflow IR intermediate representation includes: Parse the original workflow, extract the node set, edge set, and attribute set of the original workflow, and extract the node type, input parameters, and metadata for each node; Iterate through the input parameters of each node, identify static parameters, generate operators for static parameters, and add static constants to the constant set of the AI-Workflow IR module; Based on the relationship between native nodes and IR operators of the AI-Workflow IR module, each node is mapped to an IR operator based on its node type; Traverse the edge set of the native workflow, identify whether the input parameter of a node is a node reference. If so, use the output of the IR operator of the parent node of the node reference as the input of the IR operator of the lower node. If the input parameter of the node is identified as a static parameter, use the static constant corresponding to the node as the input of the IR operator. Based on data dependencies, all operators are written into the @main function, metadata is added to each operator, and a complete standardized AI-Workflow IR intermediate representation is generated.

Images