Workflow-based edge intelligent agent deployment and scheduling method and system

Abstract

The application relates to a kind of edge intelligent agent deployment and scheduling method and system based on workflow, it includes the execution flow of intelligent agent is modeled as directed acyclic workflow graph, each subtask carries the capability demand vector consisting of task type, computing resource demand, dependency model type and time delay constraint;With preset deployment period as granularity, with the system utility function constructed by request processing amount, request transfer penalty amount and mirror deployment cost weighting as target, solve the updated intelligent agent mirror deployment scheme under the constraint of storage capacity, computing power and mirror coverage;With preset scheduling time slot as granularity, output subtask scheduling decision and mirror cache update decision through double Actor centralized Critic deep reinforcement learning network;Delay and cost virtual queue is constructed, and based on Lyapunov function, the dynamic weight closed-form solution is derived to adjust the reward function weight in real time;Through experience replay online training and periodic deployment optimization.

Description

Technical Field

[0001] This application relates to the field of intelligent agent technology, and in particular to a workflow-based method and system for deploying and scheduling edge intelligent agents. Background Technology

[0002] With the rapid development of 5G mobile communication technology and the Internet of Things (IoT), massive numbers of terminal devices are connecting to networks and generating huge amounts of real-time data at the edge. Generative artificial intelligence (AI) technologies, represented by large-scale language models, have made breakthrough progress, leading to the emergence of the concept of AI agents. An AI agent is an intelligent entity with environmental perception, autonomous planning, tool invocation, and execution feedback capabilities. It can break down complex tasks into multiple sub-tasks and autonomously orchestrate execution processes. Deploying AI agents in edge environments, enabling intelligent applications to make real-time decisions on the spot where data is generated, is a crucial development direction in the current field of edge intelligence.

[0003] Currently, the mainstream edge task offloading and service caching methods based on multi-agent deep reinforcement learning treat multiple edge devices as independent reinforcement learning agents. Each agent maintains an Actor-Critic network and outputs offloading decisions for computational tasks and cache update decisions for the service environment based on local observations. A centralized Critic network is used to evaluate the global state value and coordinate the policy optimization direction of the multiple agents. A fixed-weight multi-objective reward function is used to minimize task execution latency. In each scheduling slot, an offloading location is selected for the arriving task, and the service environment cache on the edge device is updated according to the LRU rule within a fixed deployment cycle.

[0004] The aforementioned technology employs pre-determined passive cache update rules, updating the service environment cache only within a fixed deployment cycle. When the popularity of task requests changes, the service environment on the edge device cannot match the new request pattern in a timely manner, resulting in a large number of tasks needing to be unloaded across nodes, increasing transmission latency and reducing system resource utilization.

[0005] Based on this, this application provides a workflow-based method and system for deploying and scheduling edge intelligent agents. Summary of the Invention

[0006] To address the issue that using pre-defined passive cache update rules only updates the service environment cache within a fixed deployment cycle, and that when task request popularity changes, the service environment on the edge device cannot promptly match the new request pattern, resulting in a large number of tasks needing to be unloaded across nodes, increasing transmission latency and reducing system resource utilization, this application provides a workflow-based edge agent deployment and scheduling method and system.

[0007] Firstly, this application provides a workflow-based edge agent deployment and scheduling method, which adopts the following technical solution: including: The execution flow of the intelligent agent is obtained and modeled as a directed acyclic workflow graph containing multiple subtasks. Each subtask carries a capability requirement vector, which includes task type, computing resource requirements, dependency model type and latency constraints. Based on a preset deployment cycle, the system collects operational statistics of each edge node in the previous cycle. The operational statistics include the number of requests, request transfers, average execution latency, and current deployment scheme for each type of intelligent agent image. With the goal of maximizing the system utility function, the system solves for the updated deployment scheme under the constraints of storage capacity, computing power, and image coverage, and then executes it. According to the preset scheduling time slots, multiple edge nodes that are geographically close and have similar business models are bound into a scheduling agent; each scheduling agent, through a pre-trained deep reinforcement learning network, outputs scheduling decisions for arriving sub-tasks and update decisions for local image caches based on the current local observation state and the current deployment scheme. Construct a latency virtual queue and a cost virtual queue to record the cumulative deviation of subtask execution latency exceeding the latency threshold and the cumulative deviation of operating cost exceeding the cost threshold, respectively; construct a Lyapunov function based on the two virtual queues, and derive the dynamic weight closed-form solution by minimizing the weighted sum; before the start of the next scheduling slot, update the weight coefficients of the latency and cost terms in the deep reinforcement learning network reward function according to the dynamic weight closed-form solution. After each scheduling time slot ends, the state transition data of the current time slot is stored in the experience replay buffer, and samples are taken from it to train the deep reinforcement learning network online; at the end of each deployment cycle, the running statistics of each edge node in the current cycle are collected to trigger a new round of deployment optimization.

[0008] Preferably, the objective of maximizing the system utility function includes: The system utility function is constructed by weighting the request processing volume, the request transfer penalty, and the image deployment cost. For any type of agent image deployed on any edge node, the request processing volume is calculated by the number of requests and request transfers of the corresponding type of agent image on the edge node, as well as the average execution latency of the agent image of that type on the edge node. When the number of requests for any type of agent image on an edge node exceeds a preset threshold and the corresponding edge node has not currently deployed an agent image of that type, the request transfer penalty is equal to the product of the penalty coefficient and the number of requests; otherwise, the request transfer penalty is zero. The system utility function is obtained by combining the first weighted sum of the request processing volume of each type of intelligent agent image on each edge node, the second weighted sum of the request transfer penalty volume, and the third weighted value of the image deployment cost.

[0009] Preferably, the step of solving for and executing the updated deployment scheme under constraints of storage capacity, computing power, and image coverage includes: The storage capacity constraint is that the sum of the storage space of all types of intelligent agent images deployed on any edge node does not exceed the maximum storage capacity of that edge node. The computing power constraint is that the sum of the computing resource requirements of all types of intelligent agent images deployed on any edge node does not exceed the maximum computing power of that edge node. The mirror coverage constraint is that the number of nodes deployed in the entire edge network for any type of intelligent agent mirror is not less than the preset minimum coverage number corresponding to that type of intelligent agent mirror; Define a deployment variable, which represents whether any type of intelligent agent image is deployed on any edge node. The deployment variable is a binary variable. Using the deployment variable as the decision variable, the system utility function is transformed into an objective function with the deployment variable as the independent variable. Under the constraints of storage capacity, computing power, and mirror coverage, the system utility function is modeled as a mixed-integer linear programming problem. The mixed-integer linear programming problem is solved by the branch and bound method to obtain the optimal values ​​of the deployment variables and the updated deployment scheme. Based on the differences between the updated deployment plan and the current deployment plan, newly added agent images are pulled and deployed, while resources are released for removed agent images.

[0010] Preferably, before binding multiple geographically proximate and similarly operating edge nodes into a single scheduling agent according to a preset scheduling time slot, the method further includes: Obtain the resource status of each edge node, including the maximum storage capacity, maximum computing power, and currently available resources of each edge node; Based on the capability requirement vector and the resource status, with the goal of maximizing the system utility function, and under the constraints of storage capacity, computing power and image coverage, an initial deployment scheme is obtained. According to the initial deployment scheme, images of various types of intelligent agents are pulled from the cloud and deployed to the corresponding edge nodes. A simulation environment is set up to simulate an edge network composed of multiple edge nodes, the subtask arrival process, and the initial deployment scheme. In the simulation environment, each scheduling agent interacts with the simulation environment in multiple rounds. In each round of interaction, the scheduling agent iteratively updates the parameters of the deep reinforcement learning network based on the state and reward feedback from the simulation environment until the reward value converges. The parameters of the converged deep reinforcement learning network are used as pre-trained network parameters and loaded into each scheduling agent for online workflow scheduling.

[0011] Preferably, the step of binding multiple geographically proximate and business-mode-similar edge nodes into a single scheduling agent according to a preset scheduling time slot, wherein each scheduling agent, through a pre-trained deep reinforcement learning network, outputs scheduling decisions for arriving sub-tasks and update decisions for local image caches based on the current local observation state and the current deployment scheme, including: The edge network coverage area is divided into multiple functional areas according to function. Edge nodes in the same functional area meet the conditions that the geographical distance is lower than the first preset threshold and the correlation of historical request distribution is higher than the second preset threshold. The functional areas include office areas, residential areas and commercial areas. Bind multiple edge nodes located in the same functional area to the same scheduling agent; The current local observation status includes: queue information of subtasks to be scheduled on each edge node within the area under the jurisdiction of the scheduling agent, the remaining computing resources and remaining storage resources of each edge node, the set of agent image types currently deployed on each edge node, and the link bandwidth status between the scheduling agent and neighboring scheduling agents. The current local observation state and the current deployment scheme are input into the deep reinforcement learning network, and the deep reinforcement learning network outputs the scheduling decision and the update decision; The scheduling decision is to schedule each subtask and execution position that arrives at the moment. The execution position includes: execution at the edge node of the local area, execution at the edge node of the neighboring area, execution in the cloud, execution locally after migrating the image from the neighboring area, and execution after unloading to the neighboring area and synchronously migrating the image. The update decision is to update the image of each agent cached locally. The update operation includes: keeping it unchanged, adding an instance, deleting an instance, and replacing an instance. The scheduling decision and the update decision are executed, and the subtask execution delay generated by the scheduling decision and the mirror deployment cost generated by the update decision are fed back to the deep reinforcement learning network as reward signals.

[0012] Preferably, the step of constructing a Lyapunov function based on two virtual queues and deriving a dynamic weighted closed-form solution by minimizing the weighted sum includes: At the end of each scheduling time slot, obtain the average execution latency and system operating cost of the current time slot; The latency virtual queue is updated based on the deviation between the average execution latency of the time slot and the latency threshold, and the cost virtual queue is updated based on the deviation between the system operating cost and the cost threshold. When the deviation is positive, the virtual queue length increases cumulatively, and when the deviation is negative, the virtual queue length decreases, and the virtual queue length is not less than zero. The Lyapunov function is constructed using the latency virtual queue length and the cost virtual queue length, and the Lyapunov function is the sum of the squares of the two virtual queue lengths; Calculate the single-step drift of the Lyapunov function from the previous time slot to the current time slot, and use the weighted sum of the single-step drift and the weighted optimization objective as the drift penalty objective function; Minimize the drift penalty objective function to obtain the dynamic weight closed-form solution. In the dynamic weight closed-form solution, the weight coefficient of the delay term is positively correlated with the length of the delay virtual queue, and the weight coefficient of the cost term is positively correlated with the length of the cost virtual queue. Before the start of the next scheduling slot, the weight coefficients of the delay and cost terms in the reward function of the deep reinforcement learning network are updated based on the dynamic weight closed-form solution.

[0013] Preferably, after each scheduling time slot ends, the state transition data of the current time slot is stored in the experience replay buffer, and samples are taken from it to train the deep reinforcement learning network online. At the end of each deployment cycle, the running statistics of each edge node in the current cycle are collected to trigger a new round of deployment optimization, including: After each scheduling time slot ends, the state transition data of the current time slot is stored in the experience replay buffer. The state transition data includes the current local observation state, the scheduling decision, the update decision, the reward value obtained after executing the scheduling decision and the update decision, and the local observation state of the next scheduling time slot. A batch of state transition data is randomly sampled from the experience replay buffer, the loss function is calculated using the sampled data, and the parameters of the deep reinforcement learning network are updated by gradient descent to complete one online training. At the end of each deployment cycle, the number of requests, the number of request transfers, and the average execution latency of each type of agent image on each edge node during the cycle are counted to generate the current cycle's running statistics and trigger a new round of agent image deployment optimization.

[0014] Secondly, this application provides a workflow-based edge agent deployment and scheduling device, which adopts the following technical solution, including: The process modeling module is used to obtain the execution flow of the intelligent agent and model it as a directed acyclic workflow graph containing multiple sub-tasks. Each sub-task carries a capability requirement vector, which includes task type, computing resource requirements, dependency model type and latency constraints. The deployment module is used to collect the operational statistics of each edge node in the previous period based on a preset deployment cycle. The operational statistics include the number of requests, request transfers, average execution latency, and current deployment scheme for each type of intelligent agent image. With the goal of maximizing the system utility function, under the constraints of storage capacity, computing power, and image coverage, the module solves for the updated deployment scheme and executes it. The decision update module is used to bind multiple edge nodes that are geographically close and have similar business models into a scheduling agent according to the preset scheduling time slots. Each scheduling agent outputs scheduling decisions for arriving sub-tasks and update decisions for local image caches based on the current local observation state and the current deployment scheme through a pre-trained deep reinforcement learning network. The weight update module is used to construct a latency virtual queue and a cost virtual queue, which respectively record the cumulative deviation of subtask execution latency exceeding the latency threshold and the cumulative deviation of operating cost exceeding the cost threshold; based on the two virtual queues, a Lyapunov function is constructed, and the dynamic weight closed-form solution is derived by minimizing the weighted sum; before the start of the next scheduling slot, the weight coefficients of the latency term and the cost term in the deep reinforcement learning network reward function are updated according to the dynamic weight closed-form solution; The iterative training module is used to store the state transition data of the current time slot into the experience replay buffer after each scheduling time slot ends, and to sample from it to train the deep reinforcement learning network online; at the end of each deployment cycle, it collects the running statistics of each edge node in the current cycle to trigger a new round of deployment optimization.

[0015] Thirdly, this application also provides a control device, the device comprising: It includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed, such as the workflow-based edge agent deployment and scheduling method described above.

[0016] Fourthly, this application also provides a computer-readable storage medium storing a computer program that can be loaded by a processor and executed as described above regarding the workflow-based edge agent deployment and scheduling method.

[0017] In summary, this application models the execution flow of the AI ​​agent as a directed acyclic workflow graph containing multiple subtasks. Each subtask carries a capability requirement vector consisting of task type, computational resource requirements, dependency model type, and latency constraints, enabling the system to perceive the internal structure and differentiated requirements of the workflow. Based on this, with a preset deployment cycle as the time granularity, the system collects the operational statistics of each edge node in the previous cycle. With the objective of maximizing the system utility function weighted by request processing volume, request transfer penalty, and image deployment cost, the system solves for and executes an updated agent image deployment scheme under constraints of storage capacity, computational power, and image coverage. At the micro-timescale, using preset scheduling slots as the granularity, multiple geographically proximate edge nodes with similar business models are bound together as a scheduling agent. Each scheduling agent, through a pre-trained deep reinforcement learning network, outputs scheduling decisions for arriving subtasks and update decisions for local image caches based on the current local observation state and deployment plan. The scheduling decisions support various methods, including local execution, neighbor region execution, cloud execution, local execution after migrating an image from a neighbor, and offloading execution with simultaneous image migration, achieving collaborative decision-making between scheduling and caching. Simultaneously, latency virtual queues and cost virtual queues are constructed to record the accumulated deviations exceeding their respective thresholds. A dynamic weight closed-form solution is derived based on Lyapunov functions, and the weight coefficients of the latency and cost terms in the reward function are updated in real-time during each scheduling slot, achieving adaptive adjustment of the optimization direction. Online training is performed through experience replay after each scheduling slot ends, and a new round of deployment optimization is triggered at the end of each deployment cycle. This approach enables fine-grained scheduling of AI agent execution processes through workflow modeling. By combining scheduling and caching in collaborative decision-making, it overcomes the limitations of separating caching and scheduling strategies in traditional methods. Through a dynamic weighting mechanism, it achieves an adaptive balance between latency and cost. The two-layer architecture balances global deployment optimization with real-time scheduling response capabilities, thus solving the problem of efficient deployment and scheduling of AI agents in resource-constrained edge environments. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a workflow-based method for deploying and scheduling edge agents.

[0019] Figure 2 This is a structural block diagram of a workflow-based edge agent deployment and scheduling device. Detailed Implementation

[0020] The following combination Figures 1-2 This application will be described in further detail.

[0021] This application provides a workflow-based edge agent deployment and scheduling scheme. The execution entity is the agent control system, and the system architecture adopts a four-level deployment: the terminal layer is responsible for initiating agent task requests and providing local perception and execution capabilities; the edge layer is deployed on the edge server on the base station or gateway side, carrying the operation of agent instances and providing low-latency computing and inference services for sub-tasks; the cloud layer deploys large models and a global knowledge base to support complex planning tasks in the agent workflow; and the management and orchestration layer acts as a global controller, responsible for parsing the agent workflow, periodically deploying and optimizing agent images, and online scheduling of the workflow execution process.

[0022] Reference Figure 1 The embodiments of this application include at least steps S10 to S50.

[0023] S10, obtain the execution flow of the intelligent agent and model it as a directed acyclic workflow graph containing multiple subtasks. Each subtask carries a capability requirement vector, which includes task type, computing resource requirements, dependency model type and latency constraints. S20: Based on the preset deployment cycle, collect the operation statistics of each edge node in the previous cycle. The operation statistics include the number of requests, the number of request transfers, the average execution latency, and the current deployment scheme for each type of intelligent agent image. With the goal of maximizing the system utility function, under the constraints of storage capacity, computing power, and image coverage, solve for the updated deployment scheme and execute it. S30: According to the preset scheduling time slot, multiple edge nodes that are geographically close and have similar business models are bound into a scheduling agent; each scheduling agent, through a pre-trained deep reinforcement learning network, outputs scheduling decisions for arriving sub-tasks and update decisions for local image caches based on the current local observation state and the current deployment scheme. S40, construct a latency virtual queue and a cost virtual queue to record the cumulative deviation of subtask execution latency exceeding the latency threshold and the cumulative deviation of operating cost exceeding the cost threshold, respectively; construct a Lyapunov function based on the two virtual queues, and derive the dynamic weight closed-form solution by minimizing the weighted sum; before the start of the next scheduling slot, update the weight coefficients of the latency and cost terms in the deep reinforcement learning network reward function according to the dynamic weight closed-form solution; S50: After each scheduling time slot ends, the state transition data of the current time slot is stored in the experience replay buffer, and samples are taken from it to train the deep reinforcement learning network online; at the end of each deployment cycle, the running statistics of each edge node in the current cycle are collected to trigger a new round of deployment optimization.

[0024] Among them, geographical proximity and similar business models can be understood as geographical distance being lower than the first preset threshold and the correlation of historical request distribution being higher than the second preset threshold.

[0025] Specifically, in step S10, based on the task description of the agent, multiple subtasks and the dependencies between subtasks are parsed; a directed acyclic workflow graph is constructed with subtasks as nodes and dependencies as directed edges, and each directed edge carries the data transfer constraints between the two subtasks it connects; wherein, the computational resource requirements in the capability requirement vector are estimated based on the task type and dependency model type of the corresponding subtask.

[0026] Step S10 models the agent's execution process as a directed acyclic workflow graph. Each subtask carries a capability requirement vector containing task type, computational resource requirements, dependency model type, and latency constraints. This allows the scheduling system to perceive the internal structure of the workflow and the differentiated requirements of each subtask, enabling fine-grained scheduling. Steps S20 and S30 fuse the subtask scheduling decisions with the agent's mirror cache update decisions into the parallel output of the same deep reinforcement learning network. The scheduling decisions support proactive scheduling methods such as local execution after migrating a mirror from a neighbor and offloading execution while synchronously migrating a mirror. This allows cache updates and task scheduling to make collaborative decisions within each scheduling slot, achieving real-time response of mirror deployment to changes in request patterns. Step S40 constructs a latency virtual queue and a cost virtual queue to record the accumulated deviations exceeding their respective thresholds. Based on the Lyapunov function, a dynamic weight closed-form solution is derived. In each scheduling slot, the weight coefficients of the latency and cost terms in the reward function are updated in real time, allowing the optimization direction to adaptively adjust with the system state.

[0027] In some embodiments, step S20 specifically includes the following steps: For any type of agent image deployed on any edge node, the request processing volume is calculated from the number of requests, the number of request transfers, and the average execution latency of the agent image of that type on the edge node; when the number of requests for any type of agent image on an edge node exceeds a preset threshold and the corresponding edge node does not currently deploy an agent image of that type, the request transfer penalty is equal to the product of the penalty coefficient and the number of requests; otherwise, the request transfer penalty is zero; the system utility function is obtained by combining the first weighted sum of the request processing volume of each type of agent image on each edge node, the second weighted sum of the request transfer penalty, and the third weighted value of the image deployment cost.

[0028] The system utility function is constructed by weighting request processing volume, request transfer penalty, and image deployment cost, and is used to quantify the overall performance of the current deployment scheme.

[0029] Specifically, request processing volume The calculation is based on the actual number of service requests for each type of image on each node, the number of requests that could not be processed locally and were moved out, and the average execution latency. The formula is as follows: ; in, This is a weighting coefficient, which takes the value of a positive real number. It is used to adjust the magnitude of the request processing volume in the system utility function. It is usually normalized based on the maximum number of requests processed per unit time slot of a single node in the system. This represents the unit delay, taken as the length of a scheduling time slot, with time as its dimension. Its purpose is to smooth the denominator, preventing the calculation results from diverging when the average execution delay approaches zero, and simultaneously unifying the dimensions of all terms in the denominator. Let n be the number of requests to mirror an agent of type m on edge node n. This represents the number of requested transfers for this image on node n. This represents the average execution latency of the image on node n, reflecting the actual service efficiency of the image at its deployment location per unit latency. The request transfer penalty incentivizes the system to proactively allocate resources to nodes with high request volumes but no corresponding image deployed. When the number of requests for a certain type of image on a node exceeds a preset threshold and the node has not yet deployed that image, a penalty is applied by multiplying the penalty coefficient by the number of requests; otherwise, it is zero. The image deployment cost reflects the operational overhead required to maintain each image instance. Therefore, the system utility function... It can be represented as: ; in, , , These are the first weight, the second weight, and the third weight, respectively. All three weights are positive real numbers, and their sum is 1. For the set of edge nodes, A collection of agent image types. Let m be the request transfer penalty for a mirror image of type m on node n. The image deployment cost for deploying an image of a type m agent on edge node n. This serves as a deployment variable. The above construction method enables the system utility function to simultaneously measure the service capacity of the deployment scheme, the rationality of resource allocation, and the operating cost, providing a unified quantitative target for solving the optimal deployment scheme.

[0030] Furthermore, step S20 also includes the following steps: Storage capacity constraint: the sum of storage space for all types of intelligent agent images deployed on any edge node does not exceed the maximum storage capacity of that edge node; Computational capability constraint: the sum of computational resource requirements for all types of intelligent agent images deployed on any edge node does not exceed the maximum computational capability of that edge node; Image coverage constraint: the number of nodes where any type of intelligent agent image is deployed in the entire edge network is not less than the preset minimum coverage number corresponding to that type of intelligent agent image; Defining deployment variables: the deployment variables represent whether any type of intelligent agent image is deployed on any edge node, and the deployment variables are binary variables; Using the deployment variables as decision variables, the system utility function is transformed into an objective function with the deployment variables as independent variables; Under the storage capacity constraint, computational capability constraint, and image coverage constraint, maximizing the system utility function is modeled as a mixed-integer linear programming problem, and the mixed-integer linear programming problem is solved using the branch and bound method to obtain the optimal value of the deployment variables, resulting in an updated deployment scheme; Based on the difference between the updated deployment scheme and the current deployment scheme, pull deployment is performed on newly added intelligent agent images, and resource release is performed on removed intelligent agent images.

[0031] Specifically, when solving for the updated deployment scheme, a binary deployment variable is introduced. Indicates whether an agent image of type m is deployed on edge node n, transforming the system utility function into... The objective function is defined by the independent variable, and the solution process is subject to three types of constraints: Storage capacity constraints are: ; Wherein, the subscript m is the agent image type index, used to distinguish different agent image types. The storage space requirements for mirror m. This represents the maximum storage capacity of node n. The computational capability constraints are: ; in, The computational resource requirements for mirror m. The maximum computing power of node n; The mirror coverage constraint is: ; in, This is the preset minimum coverage number for the image of an agent of type m.

[0032] Under this constraint, the optimization problem of maximizing the system utility function is modeled as a mixed-integer linear programming problem, and solved using the branch and bound method to obtain the optimal values ​​of each deployment variable, forming an updated deployment scheme. The updated scheme is compared with the current scheme; newly added images are deployed by pull, and removed images have their occupied resources released. Thus, through mathematical programming, a globally optimal solution for agent image deployment under multiple constraints is achieved, maximizing the overall service efficiency of the system while ensuring resource constraints and coverage requirements.

[0033] In some embodiments, considering the initial deployment and pre-training of the deep learning network, the corresponding processing steps are as follows: Obtain the resource status of each edge node, including its maximum storage capacity, maximum computing power, and currently available resources; based on the capability requirement vector and resource status, with the goal of maximizing the system utility function, solve for the initial deployment scheme under constraints of storage capacity, computing power, and image coverage; pull images of various types of agents from the cloud and deploy them to the corresponding edge nodes according to the initial deployment scheme; build a simulation environment that simulates the edge network composed of multiple edge nodes, the sub-task arrival process, and the initial deployment scheme; in the simulation environment, each scheduling agent interacts with the simulation environment in multiple rounds, iteratively updating the parameters of the deep reinforcement learning network based on the status and reward feedback from the simulation environment in each round until the reward value converges; use the converged parameters of the deep reinforcement learning network as pre-trained network parameters and load them into each scheduling agent for online workflow scheduling.

[0034] Specifically, the resource status of each edge node is acquired, and combined with the capability requirement vector, the initial deployment scheme is solved under the constraints of storage capacity, computing power, and image coverage, with the goal of maximizing the system utility function. Images of various types of agents are pulled from the cloud and deployed to the corresponding edge nodes, enabling the system to obtain a runnable initial state. Subsequently, a simulation environment is built to simulate the edge network topology, subtask arrival process, and initial deployment scheme. Each scheduling agent interacts in the simulation environment for multiple rounds, iteratively updating the deep reinforcement learning network parameters based on the state and reward feedback from the environment until the reward value converges. The converged parameters are then loaded as pre-trained network parameters into each scheduling agent for subsequent online scheduling. This solves the cold start problem of deep reinforcement learning networks, and through simulation pre-training, the network obtains a reasonable initial strategy before formal operation, reducing scheduling performance fluctuations caused by random exploration in the early stages of online operation.

[0035] In some embodiments, step S30 specifically includes the following steps: dividing the edge network coverage area into multiple functional areas, including office areas, residential areas, and commercial areas; binding multiple edge nodes located in the same functional area into the same scheduling agent; the current local observation state includes: queue information of subtasks to be scheduled on each edge node in the area under the jurisdiction of the scheduling agent, the remaining computing resources and remaining storage resources of each edge node, the set of agent image types currently deployed on each edge node, and the link bandwidth status between the scheduling agent and neighboring scheduling agents; inputting the current local observation state and the current deployment scheme into the deep reinforcement learning network, and the deep reinforcement learning network outputs scheduling decisions and update decisions; executing the scheduling decisions and update decisions, and feeding back the subtask execution latency generated by the scheduling decisions and the image deployment cost generated by the update decisions as reward signals to the deep reinforcement learning network.

[0036] The scheduling decision involves scheduling each arriving subtask and its execution location. Execution locations include: execution at the local edge node, execution at the neighboring edge node, execution in the cloud, execution locally after migrating an image from a neighboring region, and execution by unloading to a neighboring region while simultaneously migrating the image. The update decision involves updating each locally cached agent image, including: keeping it unchanged, adding an instance, deleting an instance, and replacing an instance.

[0037] The deep reinforcement learning network employs a dual-Actor-centralized Critic architecture, comprising: an offloading decision Actor network, which outputs the execution positions of each subtask in the scheduling decision based on the current local observation state; a cache update Actor network, which outputs the addition, deletion, or replacement operations of each type of agent's image in the update decision based on the current local observation state; and a centralized Critic network, which receives the joint local observation state of all scheduling agents and outputs a global state value estimate to guide the policy optimization direction of the offloading decision Actor network and the cache update Actor network. Both Actor networks use a DuelingDQN structure, with the hidden layers of the network outputting state value functions. and action advantage function Then, the Q-values ​​for each action are obtained by combining the two using the following formula: ; in, For the input state, For action, Let be the state value under state s. The advantage of performing action a in state s. For the action space, This is to traverse every action in the action space. To perform an action in state s The above structure separates state value estimation from action advantage estimation, thus improving the accuracy of Q-value estimation.

[0038] The initial construction form of the reward signal is as follows: the ratio of the subtask execution latency to the preset maximum latency value is used as the normalized latency term, the ratio of the image deployment cost to the preset maximum cost value is used as the normalized cost term, and the weighted sum of the normalized latency term and the normalized cost term is used as the reward value, that is: ; in, As a reward value, For subtask execution delay, The preset maximum delay value, For the cost of mirror deployment, The preset maximum cost value, and These are the weighting coefficients for the delay and cost items, respectively; weighting coefficients and The initial value is set to be equal, and it is dynamically adjusted in step S40 during subsequent scheduling. The reward value is calculated using an online normalization method, that is, the mean and variance of latency and cost are dynamically calculated in the data stream using the Welford algorithm. Based on the current mean and variance, the latency and cost terms are standardized in real time to cope with the characteristics of continuous data generation and unknown distribution in edge environments.

[0039] Specifically, the edge network coverage area is divided into functional regions based on business characteristics. Multiple edge nodes with similar business models within the same functional region are bound together as a scheduling agent, enabling scheduling decisions to leverage the homogeneity of business within the region to improve efficiency. The scheduling agent collects the current local observation state in each scheduling time slot, including the queue of subtasks to be scheduled for each node within its jurisdiction, the remaining computing and storage resources, the set of deployed agent image types, and the link bandwidth status with neighboring scheduling agents. This state, along with the current deployment plan, is input into a deep reinforcement learning network. The offloading decision Actor network and the cache update Actor network output scheduling and update decisions in parallel: the scheduling decision allocates an execution position for each subtask from five options: execution in the local region, execution in a neighboring region, execution in the cloud, execution locally after migrating an image, or offloading and synchronously migrating an image; the update decision selects an operation for each locally cached image from adding, deleting, replacing, or keeping it unchanged. After execution, the subtask execution latency and image deployment cost are normalized as described above to construct a reward signal, which is fed back to the centralized Critic network to calculate the global state value estimate, thereby guiding the policy updates of the two Actor networks. By separating different types of decisions through dual actors, coordinating global decisions through centralized Critic, and integrating scheduling and caching into a unified decision-making and proactive image migration capability, the system can dynamically adjust image distribution while responding to subtask requests in real time, effectively improving edge resource utilization efficiency and workflow execution performance.

[0040] In some embodiments, step S40 specifically includes the following steps: at the end of each scheduling time slot, obtain the average execution latency and system operating cost of the current time slot; update the latency virtual queue according to the deviation between the average execution latency and the latency threshold, and update the cost virtual queue according to the deviation between the system operating cost and the cost threshold. When the deviation is positive, the virtual queue length increases cumulatively; when the deviation is negative, the virtual queue length decreases, and the virtual queue length is not less than zero; construct a Lyapunov function with the latency virtual queue length and the cost virtual queue length, where the Lyapunov function is the sum of the squares of the two virtual queue lengths; Calculate the single-step drift of the Lyapunov function from the previous time slot to the current time slot, and use the weighted sum of the single-step drift and the weighted optimization objective as the drift penalty objective function. Minimize the drift penalty objective function to obtain the dynamic weight closed-form solution. In the dynamic weight closed-form solution, the weight coefficient of the delay term is positively correlated with the length of the delay virtual queue, and the weight coefficient of the cost term is positively correlated with the length of the cost virtual queue. Before the start of the next scheduling time slot, update the weight coefficients of the delay term and the cost term in the reward function of the deep reinforcement learning network based on the dynamic weight closed-form solution.

[0041] Specifically, by constructing latency virtual queues and cost virtual queues, the multi-objective optimization problem is transformed into a queue stability control problem. At the end of each scheduling slot, the average execution latency and system operating cost of the current slot are obtained and compared with latency thresholds and cost thresholds, respectively. The latency virtual queue is updated as follows: ; in, The length of the virtual queue for the current time slot t is the delay. The average execution delay per time slot. For the time delay threshold, This is the time delay sensitivity coefficient, used to adjust the tightness of the time delay constraint, with a value ranging from zero to 1. The value of determines the magnitude of the response of the virtual queue to latency deviations: The closer it is to 1, the more significant the cumulative effect of a single latency exceedance on the virtual queue, and the more sensitive the system is to latency fluctuations; The closer to 0, the more smoothly the virtual queue grows, and the more tolerant the system is to latency fluctuations.

[0042] The update method for the cost virtual queue is as follows: ; in, The cost of the virtual queue length for the current time slot t. For system operating costs, As a cost threshold, This is the cost sensitivity coefficient, ranging from zero to 1. Its value determines the response magnitude of the cost virtual queue to cost deviations.

[0043] When the actual value exceeds the threshold, the corresponding virtual queue length increases cumulatively; conversely, it decreases when the actual value falls below the threshold, and the queue length is not less than zero. Thus, the virtual queue length directly reflects the cumulative degree to which various indicators exceed the constraint threshold.

[0044] Construct the Lyapunov function using the sum of the squares of the lengths of the two virtual queues: ; This function geometrically characterizes the degree of deviation between the current state of the system and the ideal stable point. A larger value indicates a more severe cumulative delay violation and cost overrun. Calculate the single-step drift of this function from the previous time slot to the current time slot. : ; This drift reflects the changing trend of system instability. Using the weighted sum of the single-step drift and the weighted optimization objective as the drift penalty objective function, minimizing this objective function yields the dynamic weighted closed-form solution: ; in, The weighting coefficient for the delay term in time slot t. The cost term weighting coefficient for time slot t. The regularization term is used to prevent the denominator from being zero. The closed-form solution above shows that the delay term weight coefficient is positively correlated with the delay virtual queue length, and the cost term weight coefficient is positively correlated with the cost virtual queue length. Before the start of the next scheduling slot, and Update the reward function of the deep reinforcement learning network. This allows the multi-objective optimization direction to adaptively adjust according to the actual operating state of the system: when a certain indicator continuously exceeds the standard, its corresponding weight automatically increases, driving the scheduling strategy to prioritize the optimization of that indicator; when the indicator returns to normal, the weight decreases accordingly, so as to achieve a dynamic balance between latency and cost.

[0045] In some embodiments, step S50 specifically includes the following steps: after each scheduling time slot ends, the state transition data of the current time slot is stored in the experience replay buffer. The state transition data includes the current local observation state, scheduling decision, update decision, reward value obtained after executing the scheduling decision and update decision, and the local observation state of the next scheduling time slot; a batch of state transition data is randomly sampled from the experience replay buffer, the loss function is calculated using the sampled data, and the parameters of the deep reinforcement learning network are updated by gradient descent to complete one online training; at the end of each deployment cycle, the number of requests, the number of request transitions, and the average execution latency of each type of agent image on each edge node in this cycle are counted to generate the running statistics information of the current cycle, and a new round of agent image deployment optimization is triggered.

[0046] Specifically, at the end of each scheduling slot, the state transition data of the current slot—including the current local observation state, scheduling decision, update decision, reward value obtained after execution, and the local observation state of the next slot—is stored as a complete interaction experience in the experience replay buffer. Then, a batch of historical experience data is randomly sampled from the buffer, and the loss function is calculated using the sampled data. The deep reinforcement learning network parameters are then updated via gradient descent, completing one online training cycle. This allows the scheduling strategy to continuously learn and optimize from real-time interactions. At the end of each deployment cycle, the number of requests, request transitions, and average execution latency of each type of agent image on each edge node within that cycle are statistically analyzed to generate operational statistics for that cycle, triggering a new round of agent image deployment optimization. The dual-loop mechanism serves two purposes: the inner loop's online training allows network parameters to continuously evolve with environmental changes, maintaining the real-time adaptability of the scheduling strategy; the outer loop's periodic deployment optimization re-solves for the globally optimal image distribution based on long-term operational statistics, achieving a balance between short-term scheduling response and long-term resource allocation efficiency.

[0047] Furthermore, in real edge computing environments, feedback data such as the statistics of subtask execution latency, the calculation of image deployment costs, and the updates of resource status of each edge node can usually only be aggregated after the current scheduling slot ends. This results in the scheduling agent being unable to immediately obtain the actual reward signal and the local observation status of the next scheduling slot after making scheduling and update decisions in the current slot, leading to a feedback delay of one slot length. To address this data acquisition lag issue, the following adjustments are made during online training: In the current scheduling time slot t, the scheduling agent determines the current local observation state. The scheduling decision is executed by outputting a deep reinforcement learning network. and updated decision After that, instead of immediately calculating the reward and collecting the next state, it waits for a delay period of time. During the delay period, the edge network processes user requests normally; at the end of the delay period, the cumulative subtask execution latency and image deployment cost during that delay period are obtained to construct the post-delay reward value. And collect the delayed local observation status. ;Will This complete state transition data is stored in the experience replay buffer.

[0048] Among them, delay duration The length of a scheduling slot is set to ensure that the correspondence between feedback data and decision-making moments is correctly linked. Through this data latency awareness mechanism, deep reinforcement learning networks can be trained online in real-world edge environments with the correct correspondence between states, actions, and rewards, avoiding misalignment of state transition data due to data acquisition delays and ensuring the accuracy of online training.

[0049] The implementation principle of the workflow-based edge agent deployment and scheduling method in this application is as follows: The execution flow of the AI ​​agent is modeled as a directed acyclic workflow graph containing multiple sub-tasks. Each sub-task carries a capability requirement vector consisting of task type, computing resource requirements, dependency model type, and latency constraints, enabling the system to perceive the internal structure and differentiated requirements of the workflow. Based on this, with a preset deployment cycle as the time granularity, the running statistics of each edge node in the previous cycle are collected. With the goal of maximizing the system utility function constructed by weighting request processing volume, request transfer penalty volume, and mirror deployment cost, an updated agent mirror deployment scheme is obtained and executed under storage capacity, computing power, and mirror coverage constraints. At the micro-timescale, using preset scheduling slots as the granularity, multiple geographically proximate edge nodes with similar business models are bound together as a scheduling agent. Each scheduling agent, through a pre-trained deep reinforcement learning network, outputs scheduling decisions for arriving subtasks and update decisions for local image caches based on the current local observation state and deployment plan. The scheduling decisions support various methods, including local execution, neighbor region execution, cloud execution, local execution after migrating an image from a neighbor, and offloading execution with simultaneous image migration, achieving collaborative decision-making between scheduling and caching. Simultaneously, latency virtual queues and cost virtual queues are constructed to record the accumulated deviations exceeding their respective thresholds. A dynamic weight closed-form solution is derived based on Lyapunov functions, and the weight coefficients of the latency and cost terms in the reward function are updated in real-time during each scheduling slot, achieving adaptive adjustment of the optimization direction. Online training is performed through experience replay after each scheduling slot ends, and a new round of deployment optimization is triggered at the end of each deployment cycle. This approach enables fine-grained scheduling of AI agent execution processes through workflow modeling. By combining scheduling and caching in collaborative decision-making, it overcomes the limitations of separating caching and scheduling strategies in traditional methods. Through a dynamic weighting mechanism, it achieves an adaptive balance between latency and cost. The two-layer architecture balances global deployment optimization with real-time scheduling response capabilities, thus solving the problem of efficient deployment and scheduling of AI agents in resource-constrained edge environments.

[0050] Figure 1 This is a flowchart illustrating a workflow-based edge agent deployment and scheduling method in one embodiment. It should be understood that, although... Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows; unless explicitly stated otherwise, there is no strict order requirement for the execution of these steps, and they can be executed in other orders; and Figure 1At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0051] Based on the same technical concept, referring to Figure 2 This application also provides a workflow-based edge agent deployment and scheduling device, which adopts the following technical solution: the device includes: The process modeling module is used to obtain the execution flow of the intelligent agent and model it as a directed acyclic workflow graph containing multiple subtasks. Each subtask carries a capability requirement vector, which includes task type, computing resource requirements, dependency model type and latency constraints. The deployment module is used to collect the operational statistics of each edge node in the previous period based on a preset deployment cycle. The operational statistics include the number of requests, request transfers, average execution latency, and current deployment scheme for each type of intelligent agent image. With the goal of maximizing the system utility function, under the constraints of storage capacity, computing power, and image coverage, the module solves for the updated deployment scheme and executes it. The decision update module is used to bind multiple edge nodes that are geographically close and have similar business models into a scheduling agent according to the preset scheduling time slots. Each scheduling agent outputs scheduling decisions for arriving sub-tasks and update decisions for local image caches based on the current local observation state and the current deployment scheme through a pre-trained deep reinforcement learning network. The weight update module is used to construct a latency virtual queue and a cost virtual queue, which respectively record the cumulative deviation of subtask execution latency exceeding the latency threshold and the cumulative deviation of operating cost exceeding the cost threshold; based on the two virtual queues, a Lyapunov function is constructed, and the dynamic weight closed-form solution is derived by minimizing the weighted sum; before the start of the next scheduling slot, the weight coefficients of the latency and cost terms in the deep reinforcement learning network reward function are updated according to the dynamic weight closed-form solution; The iterative training module is used to store the state transition data of the current time slot into the experience replay buffer after each scheduling time slot ends, and to sample from it to train the deep reinforcement learning network online; at the end of each deployment cycle, it collects the running statistics of each edge node in the current cycle to trigger a new round of deployment optimization.

[0052] In some embodiments, the scheme deployment module is specifically used to calculate the request processing volume for any type of intelligent agent image deployed on any edge node by the number of requests, the number of request transfers, and the average execution latency of the intelligent agent image of that type on the edge node for any type of intelligent agent image deployed on any edge node. When the number of requests for any type of agent image on an edge node exceeds a preset threshold and the corresponding edge node does not currently deploy an agent image of that type, the request transfer penalty is equal to the product of the penalty coefficient and the number of requests; otherwise, the request transfer penalty is zero. The system utility function is obtained by combining the first weighted sum of the request processing volume of each type of intelligent agent image on each edge node, the second weighted sum of the request transfer penalty volume, and the third weighted sum of the image deployment cost.

[0053] In some embodiments, the scheme deployment module is specifically used to constrain the storage capacity to the point that the sum of the storage space of all types of intelligent agent images deployed on any edge node does not exceed the maximum storage capacity of that edge node. The computing power constraint is that the sum of the computing resource requirements of all types of intelligent agent images deployed on any edge node does not exceed the maximum computing power of that edge node. The mirror coverage constraint is that the number of nodes deployed in the entire edge network for any type of intelligent agent mirror is not less than the preset minimum coverage number corresponding to that type of intelligent agent mirror; Define a deployment variable, which represents whether any type of intelligent agent image is deployed on any edge node. The deployment variable is a binary variable. Using the deployment variable as the decision variable, the system utility function is transformed into an objective function with the deployment variable as the independent variable. Under the constraints of storage capacity, computing power, and mirror coverage, the system utility function is modeled as a mixed-integer linear programming problem. The mixed-integer linear programming problem is solved by the branch and bound method to obtain the optimal values ​​of the deployment variables and the updated deployment scheme. Based on the differences between the updated deployment plan and the current deployment plan, newly added agent images are pulled and deployed, while resources are released for removed agent images.

[0054] In some embodiments, the solution deployment module is further configured to obtain the resource status of each edge node, including the maximum storage capacity, maximum computing power, and currently available resources of each edge node; Based on the capability requirement vector and resource status, with the goal of maximizing the system utility function, and under the constraints of storage capacity, computing power and image coverage, the initial deployment scheme is obtained. According to the initial deployment scheme, images of various types of intelligent agents are pulled from the cloud and deployed to the corresponding edge nodes. Set up a simulation environment to simulate an edge network consisting of multiple edge nodes, the arrival process of subtasks, and the initial deployment scheme. In the simulation environment, each scheduling agent interacts with the simulation environment in multiple rounds. In each round of interaction, the scheduling agent iteratively updates the parameters of the deep reinforcement learning network based on the state and reward feedback from the simulation environment until the reward value converges. The parameters of the converged deep reinforcement learning network are used as pre-trained network parameters and loaded into each scheduling agent for online workflow scheduling.

[0055] In some embodiments, the decision update module is specifically used to divide the edge network coverage area into multiple functional areas according to function, including office areas, residential areas and commercial areas; Bind multiple edge nodes located in the same functional area to the same scheduling agent; The current local observation status includes: queue information of subtasks to be scheduled on each edge node within the area under the jurisdiction of the scheduling agent, the remaining computing resources and storage resources of each edge node, the set of agent image types currently deployed on each edge node, and the link bandwidth status between the scheduling agent and neighboring scheduling agents. The current local observation state and the current deployment plan are input into the deep reinforcement learning network, and the deep reinforcement learning network outputs scheduling decisions and update decisions. The scheduling decision is to schedule each subtask that arrives at the current time and its execution location. The execution locations include: execution at the edge node of the local region, execution at the edge node of the neighboring region, execution in the cloud, execution locally after migrating the image from the neighboring region, and execution by unloading to the neighboring region and synchronously migrating the image. The update decision is an update operation for each agent image in the local cache. The update operations include: keep it unchanged, add an instance, delete an instance, and replace an instance. The system executes scheduling and update decisions, and feeds the execution latency of subtasks generated by the scheduling decision and the mirror deployment cost generated by the update decision as reward signals to the deep reinforcement learning network.

[0056] In some embodiments, the weight update module is specifically used to obtain the average execution latency and system operating cost of the current time slot at the end of each scheduling time slot; The virtual delay queue is updated based on the deviation between the average execution delay of the time slot and the delay threshold. The virtual cost queue is updated based on the deviation between the system operating cost and the cost threshold. When the deviation is positive, the virtual queue length increases cumulatively. When the deviation is negative, the virtual queue length decreases. The virtual queue length is not less than zero. A Lyapunov function is constructed using the latency virtual queue length and the cost virtual queue length. The Lyapunov function is the sum of the squares of the two virtual queue lengths. Calculate the single-step drift of the Lyapunov function from the previous time slot to the current time slot, and use the weighted sum of the single-step drift and the weighted optimization objective as the drift penalty objective function; Minimize the drift penalty objective function to obtain a dynamic weighted closed-form solution. In the dynamic weighted closed-form solution, the weight coefficient of the delay term is positively correlated with the length of the delay virtual queue, and the weight coefficient of the cost term is positively correlated with the length of the cost virtual queue. Before the start of the next scheduling slot, the weight coefficients of the delay and cost terms in the reward function of the deep reinforcement learning network are updated based on the dynamic weight closed-form solution.

[0057] In some embodiments, the iterative training module is specifically used to store the state transition data of the current time slot into the experience replay buffer after each scheduling time slot ends. The state transition data includes the current local observation state, scheduling decision, update decision, reward value obtained after executing the scheduling decision and update decision, and the local observation state of the next scheduling time slot. A batch of state transition data is randomly sampled from the experience replay buffer. The loss function is calculated using the sampled data, and the parameters of the deep reinforcement learning network are updated by gradient descent to complete one online training. At the end of each deployment cycle, the number of requests, request transfers, and average execution latency of each type of agent image on each edge node during the cycle are counted to generate the current cycle's running statistics and trigger a new round of agent image deployment optimization.

[0058] This application also discloses a control device.

[0059] Specifically, the control device includes a memory and a processor, the memory storing a computer program that can be loaded by the processor and executed using the aforementioned workflow-based edge agent deployment and scheduling method.

[0060] This application also discloses a computer-readable storage medium.

[0061] Specifically, the computer-readable storage medium stores a computer program that can be loaded by a processor and executed as described above in the workflow-based edge agent deployment and scheduling method. The computer-readable storage medium includes, for example, various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0062] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A workflow-based method for deploying and scheduling edge agents, characterized in that, include: The execution flow of the intelligent agent is obtained and modeled as a directed acyclic workflow graph containing multiple subtasks. Each subtask carries a capability requirement vector, which includes task type, computing resource requirements, dependency model type and latency constraints. Based on a preset deployment cycle, the system collects operational statistics of each edge node in the previous cycle. The operational statistics include the number of requests, request transfers, average execution latency, and current deployment scheme for each type of intelligent agent image. With the goal of maximizing the system utility function, the system solves for the updated deployment scheme under the constraints of storage capacity, computing power, and image coverage, and then executes it. Based on the preset scheduling time slots, multiple edge nodes that are geographically close and have similar business models are bound together into a scheduling agent; Each scheduling agent, through a pre-trained deep reinforcement learning network, outputs scheduling decisions for arriving subtasks and update decisions for local image caches based on the current local observation state and the current deployment scheme. Construct a latency virtual queue and a cost virtual queue to record the cumulative deviation of subtask execution latency exceeding the latency threshold and the cumulative deviation of operating cost exceeding the cost threshold, respectively; construct a Lyapunov function based on the two virtual queues, and derive the dynamic weight closed-form solution by minimizing the weighted sum; before the start of the next scheduling slot, update the weight coefficients of the latency and cost terms in the deep reinforcement learning network reward function according to the dynamic weight closed-form solution; After each scheduling time slot ends, the state transition data of the current time slot is stored in the experience replay buffer, and samples are taken from it to train the deep reinforcement learning network online; at the end of each deployment cycle, the running statistics of each edge node in the current cycle are collected to trigger a new round of deployment optimization.

2. The workflow-based edge agent deployment and scheduling method according to claim 1, characterized in that, The objective of maximizing the system utility function includes: The system utility function is constructed by weighting the request processing volume, the request transfer penalty, and the image deployment cost. For any type of agent image deployed on any edge node, the request processing volume is calculated by the number of requests and request transfers of the corresponding type of agent image on the edge node, as well as the average execution latency of the agent image of that type on the edge node. When the number of requests for any type of agent image on an edge node exceeds a preset threshold and the corresponding edge node has not currently deployed an agent image of that type, the request transfer penalty is equal to the product of the penalty coefficient and the number of requests; otherwise, the request transfer penalty is zero. The system utility function is obtained by combining the first weighted sum of the request processing volume of each type of intelligent agent image on each edge node, the second weighted sum of the request transfer penalty volume, and the third weighted value of the image deployment cost.

3. The workflow-based edge agent deployment and scheduling method according to claim 2, characterized in that, The process of solving for and executing the updated deployment scheme under constraints of storage capacity, computing power, and image coverage includes: The storage capacity constraint is that the sum of the storage space of all types of intelligent agent images deployed on any edge node does not exceed the maximum storage capacity of that edge node. The computing power constraint is that the sum of the computing resource requirements of all types of intelligent agent images deployed on any edge node does not exceed the maximum computing power of that edge node. The mirror coverage constraint is that the number of nodes deployed in the entire edge network for any type of intelligent agent mirror is not less than the preset minimum coverage number corresponding to that type of intelligent agent mirror; Define a deployment variable, which represents whether any type of intelligent agent image is deployed on any edge node. The deployment variable is a binary variable. Using the deployment variable as the decision variable, the system utility function is transformed into an objective function with the deployment variable as the independent variable. Under the constraints of storage capacity, computing power, and mirror coverage, the system utility function is modeled as a mixed-integer linear programming problem. The mixed-integer linear programming problem is solved by the branch and bound method to obtain the optimal values ​​of the deployment variables and the updated deployment scheme. Based on the differences between the updated deployment plan and the current deployment plan, newly added agent images are pulled and deployed, while resources are released for removed agent images.

4. The workflow-based edge agent deployment and scheduling method according to claim 1, characterized in that, Before binding multiple geographically proximate and similarly operating edge nodes into a single scheduling agent according to a preset scheduling time slot, the method further includes: Obtain the resource status of each edge node, including the maximum storage capacity, maximum computing power, and currently available resources of each edge node; Based on the capability requirement vector and the resource status, with the goal of maximizing the system utility function, and under the constraints of storage capacity, computing power and image coverage, an initial deployment scheme is obtained. According to the initial deployment scheme, images of various types of intelligent agents are pulled from the cloud and deployed to the corresponding edge nodes. A simulation environment is set up to simulate an edge network composed of multiple edge nodes, the subtask arrival process, and the initial deployment scheme. In the simulation environment, each scheduling agent interacts with the simulation environment in multiple rounds. In each round of interaction, the scheduling agent iteratively updates the parameters of the deep reinforcement learning network based on the state and reward feedback from the simulation environment until the reward value converges. The parameters of the converged deep reinforcement learning network are used as pre-trained network parameters and loaded into each scheduling agent for online workflow scheduling.

5. The workflow-based edge agent deployment and scheduling method according to claim 4, characterized in that, The process involves binding multiple geographically proximate edge nodes with similar business models into a single scheduling agent based on preset scheduling time slots. Each scheduling agent, through a pre-trained deep reinforcement learning network, outputs scheduling decisions for arriving sub-tasks and update decisions for local image caches based on the current local observation state and deployment scheme, including: The edge network coverage area is divided into multiple functional areas according to function. Edge nodes in the same functional area meet the conditions that the geographical distance is lower than the first preset threshold and the correlation of the historical request distribution is higher than the second preset threshold. The functional areas include office areas, residential areas and commercial areas. Bind multiple edge nodes located in the same functional area to the same scheduling agent; The current local observation status includes: queue information of subtasks to be scheduled on each edge node within the area under the jurisdiction of the scheduling agent, the remaining computing resources and remaining storage resources of each edge node, the set of agent image types currently deployed on each edge node, and the link bandwidth status between the scheduling agent and neighboring scheduling agents. The current local observation state and the current deployment scheme are input into the deep reinforcement learning network, and the deep reinforcement learning network outputs the scheduling decision and the update decision; The scheduling decision is to schedule each subtask and execution position that arrives at the moment. The execution position includes: execution at the edge node of the local area, execution at the edge node of the neighboring area, execution in the cloud, execution locally after migrating the image from the neighboring area, and execution after unloading to the neighboring area and synchronously migrating the image. The update decision is to update the image of each agent cached locally. The update operation includes: keeping it unchanged, adding an instance, deleting an instance, and replacing an instance. The scheduling decision and the update decision are executed, and the subtask execution delay generated by the scheduling decision and the mirror deployment cost generated by the update decision are fed back to the deep reinforcement learning network as reward signals.

6. The workflow-based edge agent deployment and scheduling method according to claim 1, characterized in that, The method of constructing a Lyapunov function based on two virtual queues and deriving a dynamic weighted closed-form solution by minimizing the weighted sum includes: At the end of each scheduling time slot, obtain the average execution latency and system operating cost of the current time slot; The latency virtual queue is updated based on the deviation between the average execution latency of the time slot and the latency threshold, and the cost virtual queue is updated based on the deviation between the system operating cost and the cost threshold. When the deviation is positive, the virtual queue length increases cumulatively, and when the deviation is negative, the virtual queue length decreases, and the virtual queue length is not less than zero. The Lyapunov function is constructed using the latency virtual queue length and the cost virtual queue length, and the Lyapunov function is the sum of the squares of the two virtual queue lengths; Calculate the single-step drift of the Lyapunov function from the previous time slot to the current time slot, and use the weighted sum of the single-step drift and the weighted optimization objective as the drift penalty objective function; Minimize the drift penalty objective function to obtain the dynamic weight closed-form solution. In the dynamic weight closed-form solution, the weight coefficient of the delay term is positively correlated with the length of the delay virtual queue, and the weight coefficient of the cost term is positively correlated with the length of the cost virtual queue. Before the start of the next scheduling slot, the weight coefficients of the delay and cost terms in the reward function of the deep reinforcement learning network are updated based on the dynamic weight closed-form solution.

7. The workflow-based edge agent deployment and scheduling method according to claim 6, characterized in that, After each scheduling time slot ends, the state transition data of the current time slot is stored in the experience replay buffer, and samples are taken from it to train the deep reinforcement learning network online. At the end of each deployment cycle, the running statistics of each edge node in the current cycle are collected to trigger a new round of deployment optimization, including: After each scheduling time slot ends, the state transition data of the current time slot is stored in the experience replay buffer. The state transition data includes the current local observation state, the scheduling decision, the update decision, the reward value obtained after executing the scheduling decision and the update decision, and the local observation state of the next scheduling time slot. A batch of state transition data is randomly sampled from the experience replay buffer, the loss function is calculated using the sampled data, and the parameters of the deep reinforcement learning network are updated by gradient descent to complete one online training. At the end of each deployment cycle, the number of requests, the number of request transfers, and the average execution latency of each type of agent image on each edge node during the cycle are counted to generate the current cycle's running statistics and trigger a new round of agent image deployment optimization.

8. A workflow-based edge agent deployment and scheduling device, characterized in that, The device includes: The process modeling module is used to obtain the execution flow of the intelligent agent and model it as a directed acyclic workflow graph containing multiple sub-tasks. Each sub-task carries a capability requirement vector, which includes task type, computing resource requirements, dependency model type and latency constraints. The deployment module is used to collect the operational statistics of each edge node in the previous period based on a preset deployment cycle. The operational statistics include the number of requests, request transfers, average execution latency, and current deployment scheme for each type of intelligent agent image. With the goal of maximizing the system utility function, under the constraints of storage capacity, computing power, and image coverage, the module solves for the updated deployment scheme and executes it. The decision update module is used to bind multiple edge nodes that are geographically close and have similar business models into a scheduling agent according to the preset scheduling time slots. Each scheduling agent outputs scheduling decisions for arriving sub-tasks and update decisions for local image caches based on the current local observation state and the current deployment scheme through a pre-trained deep reinforcement learning network. The weight update module is used to construct a latency virtual queue and a cost virtual queue, which respectively record the cumulative deviation of subtask execution latency exceeding the latency threshold and the cumulative deviation of operating cost exceeding the cost threshold; based on the two virtual queues, a Lyapunov function is constructed, and the dynamic weight closed-form solution is derived by minimizing the weighted sum; before the start of the next scheduling slot, the weight coefficients of the latency term and the cost term in the deep reinforcement learning network reward function are updated according to the dynamic weight closed-form solution; The iterative training module is used to store the state transition data of the current time slot into the experience replay buffer after each scheduling time slot ends, and to sample from it to train the deep reinforcement learning network online; at the end of each deployment cycle, it collects the running statistics of each edge node in the current cycle to trigger a new round of deployment optimization.

9. A control device, characterized in that, The device includes: A memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer program is stored that can be loaded by a processor and executed as described in any one of claims 1 to 7.

Images