A double-agent cloud-edge collaborative task scheduling method under fluctuation of computing power

Abstract

This invention discloses a dual-agent cloud-edge collaborative task scheduling method under computing power fluctuations, belonging to the field of dynamic workshop scheduling and cloud-edge collaboration technology. This method constructs a dual-agent collaborative architecture consisting of a scheduling decision agent and a resource coordination agent, decoupling the joint scheduling problem into two collaborative subsystems: manufacturing execution decision-making and environmental coordination control. The scheduling decision agent uses HGNN to perform unified modeling and feature extraction of the heterogeneous relationship between "process-machine-server," generating state embedding vectors and combining a Masking Layer mechanism to filter illegal actions, achieving feasible joint decisions on process selection, machine allocation, and pre-allocation of computing nodes. The resource coordination agent dynamically adjusts risk preference and latency penalty weights in real time, guiding the scheduling strategy to avoid high-risk nodes. The two agents adopt a centralized training and distributed execution framework. This invention can effectively cope with computing power fluctuations and link instability disturbances, significantly improving scheduling robustness and system completion efficiency.

Description

Technical Field

[0001] This invention relates to the field of dynamic workshop scheduling and cloud-edge collaboration technology under manufacturing-computing serial coupling, specifically a dual-agent cloud-edge collaborative task scheduling method under computing power fluctuations. Background Technology

[0002] With the deep integration of smart manufacturing and the Industrial Internet, the production and computing processes in flexible workshops are exhibiting deep coupling. A typical scenario is that after a workpiece completes all manufacturing processes, associated computing tasks need to be triggered immediately, and the data uploaded to edge or cloud nodes for processing. However, in this cloud-edge collaborative environment, the computing resources of edge servers are often randomly preempted by non-manufacturing background tasks, while cloud-edge communication links fluctuate due to factors such as channel fading. These uncertainties severely affect the collaborative scheduling efficiency of manufacturing-computing serial tasks, directly leading to delays in the system's maximum completion time.

[0003] Many related technologies neglect the sequential timing constraints between manufacturing and computing tasks, and single-agent algorithms converge slowly when dealing with high-dimensional action spaces, easily generating illegal decisions that violate process specifications. However, this approach suffers from the following problems in practical applications: First, a single agent needs to simultaneously cover all decision variables such as process selection, machine allocation, and computing node specification, causing its action space dimension to expand exponentially with the number of workpieces, machines, and servers, leading to slow policy training convergence and a tendency to get trapped in local optima. Second, when a single agent randomly explores in a high-dimensional space, it is highly prone to outputting illegal actions that violate established process logic and resource constraints. This not only reduces the feasibility of the scheduling scheme but also introduces execution safety hazards into the actual production process. Therefore, how to effectively decouple scheduling decisions from environmental coordination functions, while simultaneously constraining and eliminating illegal decisions at the mechanistic level, has become an urgent problem to be solved. Currently, many related technologies neglect the sequential timing constraints between manufacturing and computing tasks. When using existing single-agent reinforcement learning algorithms to solve this type of cooperative scheduling problem, the following technical bottlenecks exist in practical applications: First, a single agent needs to simultaneously cover all decision variables such as process selection, machine allocation, and computing node specification. This causes its action space dimension to expand exponentially with the increase in the number of workpieces, machines, and servers, leading to slow policy training convergence and a high susceptibility to getting trapped in local optima. Second, when a single agent conducts random exploration in a high-dimensional space, it is prone to outputting illegal actions that violate established process logic and resource constraints. This not only greatly reduces the feasibility of the scheduling scheme but also brings execution safety hazards to the actual production process. Therefore, how to effectively decouple scheduling decision-making from environmental coordination functions, while constraining and eliminating illegal decisions at the mechanistic level, has become a pressing technical challenge in this field. Summary of the Invention

[0004] To address the aforementioned issues, this invention provides a dual-agent cloud-edge collaborative task scheduling method and system under computing power fluctuations. By introducing a dual-agent DDPG (Deep Deterministic Policy Gradient) architecture, this invention decouples scheduling decisions from environmental coordination logic and combines a Masking Layer (action masking mechanism) to forcibly filter illegal actions. This achieves real-time and accurate adaptation of manufacturing processes and cloud-edge computing resources under complex computing power fluctuation environments, significantly improving the system's scheduling robustness and search efficiency.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method and system for scheduling dual-agent cloud-edge collaborative tasks under computing power fluctuations, specifically including the following steps: S1. Based on the cloud-edge collaborative task time-series correlation scheduling problem CP-CETTS under the defined computing power preemption constraint, construct a mathematical optimization model for CP-CETTS; The CP-CETTS mathematical optimization model includes: optimization objective, manufacturing-computation serial coupling constraint set, and edge node resource capacity constraint term; The cloud-edge collaborative task time-series correlation scheduling problem (CP-CETTS) under computing power preemption constraints can be defined as: There are One workpiece Need to Available machines The processing is completed on the above. Each workpiece After processing is completed, corresponding computational tasks will be generated. For each process Only one machine can be selected from its set of available machines for processing, and the processing times of the machines in the set vary. Under the premise of satisfying various constraints such as priority constraints and capacity constraints, the optimization objective of this invention is to minimize the maximum completion time. .

[0006] S1.1, The optimization objective is to minimize the maximum completion time, expressed as follows: (1); In the formula, Indicates the maximum completion time.

[0007] S1.2. Based on the optimization objective, define the set of manufacturing-computation serial coupling constraints; The manufacturing-computation serial coupling constraint set includes 5 manufacturing stage constraints and 10 computational unloading and dynamic disturbance constraints.

[0008] S1.2.1 Obtain manufacturing stage constraints, including: process completion time constraints, process sequence constraints, machine unique allocation constraints, machine processing non-overlapping constraints, and define the workpiece manufacturing completion time; (1) Process completion time constraint, the expression is as follows: (2); In the formula, Indicate process Completion time, Indicate process The start time, Indicates workpiece The One process, Indicates when the process Assigned to machine ,but ;otherwise , This represents the m-th machine. Indicate process In the machine The processing start time on the surface Indicate process A collection of compatible machines Indicates machine index; (2) Operation sequence constraint, used to ensure the order of operations within the same workpiece, is expressed as follows: (3); (3) Machine unique allocation constraint, used to ensure that each process is assigned to only one compatible machine, is expressed as follows: (4); (4) Non-overlapping machine processing constraint, used to ensure that the processing operations on the same machine do not overlap, is expressed as follows: (5); In the formula, Represents a sufficiently large positive number. Indicates the workpiece index. Indicates the process index. Indicates workpiece The The completion time of each process, Indicates workpiece The The start time of each process, Represented as 0-1 variables; (5) Define the workpiece manufacturing completion time, as shown in the following expression: (6); In the formula, Indicates workpiece Manufacturing completion time, Indicates workpiece The The completion time of each process, Indicates workpiece The number of processes, Represents the first in the set of workpieces One workpiece.

[0009] S1.2.2 Obtain computation unloading and dynamic disturbance constraints, including: unique allocation constraint for computation tasks, definition of time-varying available resources on the edge side, definition of dynamic link rate, definition of upload latency on the edge side, definition of upload latency on the cloud side, edge task start condition constraint, cloud task start condition constraint, definition of computation completion time on the edge side, definition of computation completion time on the cloud side, and definition of final completion time of the workpiece. (1) Unique assignment constraint for computation tasks, used to ensure that each computation task selects only one computation node, expressed as follows: (7); In the formula, Indicates workpiece The set of optional computing nodes corresponding to the computing task. Indicates the compute node index. Indicates when the workpiece computational tasks Unloaded to node ,but ;otherwise ; (2) Define the time-varying available resources on the edge side, as shown in the following expression: (8); In the formula, Indicates time, Indicates time The actual available computing power Indicates time The actual available memory, Indicates time The computing power that was forcibly seized Indicates time The memory that was forcibly occupied Represents edge server Total computing power Represents edge server Total memory, Represents the edge server index; (3) Define the dynamic link rate, as shown in the following expression: (9); In the formula, Indicates the actual effective transmission rate. Indicates the average effective transmission rate. This indicates the amount of fluctuation in transmission rate; (4) Define the upload latency on the edge side as follows: (10); In the formula, This represents the transmission latency from the manufacturing completion machine to the edge server for the computational task corresponding to the workpiece. Represents the first in the set of workpieces One workpiece, Indicates workpiece The amount of input data corresponding to the computation task. Indicates time Below, the effective transmission rate from the machine to the edge server; (5) Define the cloud-side upload latency as follows: (11); In the formula, Indicates workpiece The total transmission latency of the corresponding computing task from the edge server to the cloud server. This indicates the effective transmission rate from the edge server to the cloud server. Indicates workpiece Manufacturing completion time; (6) Edge-side task start condition constraints, the expression is as follows: (12); In the formula, Indicates the task is on the compute node At the beginning of Indicates workpiece The transmission latency of the corresponding computing task uploaded to the computing node; (7) Cloud-side task start condition constraints, the expression is as follows: (13); In the formula, Indicates workpiece The corresponding computing task begins execution on the edge server (referring to the moment when the edge side begins collaborative processing with the cloud). This represents the ready time function of the edge server. Indicates workpiece The corresponding computational task transmission latency from the manufactured machine to the edge server; (8) Define the completion time of edge side calculation, as shown in the following expression: (14); In the formula, Indicates workpiece The corresponding computation completion time of the computing task on the edge server. Indicates workpiece The corresponding start time of the computing task on the edge server. Indicates the actual available computing power. Indicates workpiece The computational workload of the corresponding computational task; (9) Define the cloud-side computation completion time as follows: (15); In the formula, This indicates the completion time of the computation task corresponding to the workpiece on the cloud server. Indicates workpiece The total transmission latency of the corresponding computing task from the edge server to the cloud server. This indicates the computing power of the cloud server. Indicates workpiece The amount of data returned by the corresponding computation task. This indicates the effective transmission rate from the cloud server to the edge server. (10) Define the final completion time of the workpiece, as follows: (16); In the formula, Indicates the final completion time of the workpiece. Represents a set of edge servers. Indicates the edge server index. Indicates workpiece The corresponding computation completion time of the computing task on the edge server. This indicates the completion time of the computation task corresponding to the workpiece on the cloud server. This represents the decision variable for edge server execution. This represents the decision-making variable for cloud server execution. S1.3. Define edge node resource capacity constraints based on the manufacturing-computing serial coupling constraint set; (17); In the formula, This indicates the CPU computing power required to execute the computational task corresponding to the workpiece. This indicates the memory requirements for executing the computational task corresponding to the workpiece. Indicates the actual available computing power. Indicates the actual available memory. This indicates the start time of the computation task corresponding to the workpiece on the edge server.

[0010] S2. Based on the CP-CETTS mathematical optimization model, the scheduling process is modeled as a dual-agent Markov decision process (MDP), and a dual-agent MDP model is constructed. The dual-agent system includes: a scheduling decision-making agent and a resource coordination agent; Specifically, the following steps are included: S2.1 defines a manufacturing-computing serial scheduling environment based on Markov Decision Process (MDP). In the dual-agent Markov decision process (MDP) of this invention, the environment is the core carrier for agent interaction and decision execution. First, the environment needs to define the local observation space and action space of the scheduling decision agent and the resource coordination agent. Second, based on the Markov property, the environment completes transitions from the current state according to the state transition probability. To the next state The dynamic evolution of the action and the output of instant rewards for combined actions. This guides the dual agents to continuously learn and optimize strategies.

[0011] S2.2 Based on the manufacturing-computing serial scheduling environment, construct the state characteristics of the scheduling decision-making agent and the resource coordination agent; In the dual-agent cooperative scheduling system of this invention, the state is used to characterize the real-time operational features of the current manufacturing-computing system. Unlike traditional multi-agent scheduling methods that only describe the state of processes, machines, or transportation resources, the state in this invention includes not only the operational characteristics of manufacturing processes and equipment, but also information on computing-side server resources, link quality, and dynamic disturbances. During the complete scheduling process, the system maintains a state at every moment... The global state is denoted as .in, This represents the state characteristics of the scheduling decision-making agent. This represents the state characteristics of the resource coordination agent.

[0012] S2.2.1 Based on the manufacturing-computing serial scheduling environment, set the state characteristics of the scheduling agent; In this invention The state characteristics of the scheduling decision-making agent are represented by six local feature information items: (1) Indicates in Whether all workpieces have been processed at any given time. Indicates time, Indicates workpiece The One process, Indicates the workpiece index. Indicates the process index; (2) Indicates in Current process Are they eligible to be dispatched? (3) Indicates whether the workpiece has triggered a calculation task; (4) Indicates machine At any moment The earliest time that can be allocated, express The first machine, Indicates machine index; (5) Indicates machine In the interval The proportion of busyness within; (6) This indicates the average processing time for the current process across all compatible machines.

[0013] The details are shown in Table 1: S2.2.2 Based on the manufacturing-computing serial scheduling environment, set the state characteristics of the resource coordination agent; In this invention, The state characteristics of the resource coordination agent are represented, specifically including five local feature information items: (1) Indicates the edge server at time Available computing power equals total computing power minus preempted computing power; (2) This represents the total memory minus the memory that was preempted. (3) Indicates time Computing power being taken over by background tasks; (4) Indicates time Memory occupied by background tasks; (5) Indicates the computational task at time t. The execution feedback information includes waiting, execution completion, or delay status.

[0014] S2.3 Based on the manufacturing-computing serial scheduling environment, construct the action space of the scheduling decision-making agent, the action space of the resource coordination agent, and joint actions; The agent actions are used to complete the joint control of the pre-allocated manufacturing execution and computing tasks. The agent actions are represented as follows: ,in, Indicates the action to be selected for the process to be executed. Indicates the allocation of actions by the processing equipment. This indicates the pre-assigned node action for the corresponding computation task.

[0015] The resource coordination agent action is used to output environmental risk adjustment information, and the resource coordination agent action is represented as follows: ,in, Indicates the risk penalty weight. Indicates the delay penalty weight. This represents a risk indication vector used to guide the scheduling decision-making agent to avoid high-risk nodes.

[0016] The joint action is expressed as .

[0017] In the dual-agent cooperative scheduling system of this invention, the two agents each have independent but cooperative action spaces. The actions of the scheduling decision agent are mainly used to complete the joint control of the pre-allocation of manufacturing execution and computing tasks, and its actions can be represented as follows: ,in, Indicates the action to be selected for the process to be executed. Indicates the allocation of actions by the processing equipment. This represents the pre-allocated node action for the corresponding computational task. The actions of the resource coordination agent are mainly used to output environmental risk adjustment information, and their actions can be represented as follows: ,in, Indicates the risk penalty weight. Indicates the delay penalty weight. This represents a risk indication vector used to guide the scheduling decision-making agent to avoid high-risk nodes. The system at time [time value missing] The combined action can be represented as .

[0018] S2.4. Based on the optimization objective, define the reward functions for the scheduling decision agent and the resource coordination agent; In this dual-agent cooperative scheduling system, the two agents share a global optimization objective: minimizing the system's maximum completion time while simultaneously reducing the additional costs caused by edge resource preemption and link fluctuations. Therefore, at time [time value missing], The instant reward function is defined as: in, This indicates the change in upload transmission latency. This indicates changes in queuing delay. This indicates the cost of risk exposure caused by edge preemption and link fluctuations. This represents the coefficient of variation in upload transmission latency. This represents the coefficient of variation in queuing delay. This represents the risk exposure cost coefficient. Represents the global state. Indicates the state at the next moment. Indicates the maximum completion time.

[0019] S2.5. Based on the manufacturing-computing serial scheduling environment, the action space of the scheduling decision agent, the action space of the resource coordination agent, and joint actions, define the state transition process of the MDP and complete the construction of the dual-agent MDP model. The state transition process is specifically the temporal evolution process from the current state to the next state after the scheduling decision agent and the resource coordination agent in the manufacturing-computing serial scheduling environment collaboratively generate joint actions.

[0020] S3. Based on the dual-agent MDP model, construct a dual-agent collaborative architecture consisting of a scheduling decision agent and a resource coordination agent; The dual-agent collaborative architecture is as follows: a heterogeneous graph containing process nodes, machine nodes, and server nodes is constructed for the scheduling decision agent, and a heterogeneous graph neural network (HGNN) is used to extract unified features from the nodes in the heterogeneous graph and the edge connections between nodes to generate state embedding vectors, which serve as inputs to the policy network of the scheduling decision agent; the resource coordination agent also directly uses the background load of the edge server, link status, and task execution feedback as inputs to the policy network of the resource coordination agent. The server nodes include edge server nodes and cloud server nodes; S4. Design a joint strategy based on a dual-agent cooperative architecture; The joint strategy is as follows: the candidate actions output by the scheduling decision agent need to be filtered by the Action MaskingLayer mechanism, and the filtered actions of the scheduling decision agent and the actions of the resource coordination agent are combined into a joint action, and an immediate reward function is defined. S5. Based on the dual-agent collaborative architecture and joint strategy, a centralized training and decentralized execution system for dual-agent MADDPG (Multi-Agent Deep Deterministic Policy Gradient) is constructed. Through decentralized execution in the execution interaction phase and centralized training in the update phase, the problem of time-series correlation scheduling of cloud-edge collaborative tasks under computing power preemption is solved, and the method and system for time-series correlation scheduling of cloud-edge collaborative tasks under computing power preemption are completed. Specifically, the following steps are included: S5.1 Based on a dual-agent collaborative architecture, construct the current policy network, target policy network, current value network, and target value network for the scheduling decision agent and the resource coordination agent, respectively; This invention constructs a current policy Actor network for both the scheduling decision-making agent and the resource coordination agent. Target-Policy Actor Network Current Value Critic Network and the target value Critic network ; S5.2 Interaction Phase: Based on the current policy Actor network, through distributed execution and experience collection, the interaction experience tuples are stored in the experience replay pool; During the interaction phase, the scheduling decision-making agent and the resource coordination agent respectively, based on the current time... Local observation state of the scheduling decision-making agent Local observation state of resource coordination agent Generate actions independently; The interactive experience tuple Includes: current state Next state Joint actions and instant rewards ; S5.3 Update Phase: Based on the experience replay pool, the current value network, current policy network, target value network, and target policy network are updated through centralized training; The target value network takes global state and joint actions as input; Furthermore, S5.3 specifically includes the following steps: S5.3.1 Based on the experience replay pool, the current value network and gradient information are updated by calculating the target Q-value and the value loss function; Specifically, the system extracts historical interaction samples from the experience replay pool, constructs a target value using the target policy network and the target value network, and performs gradient backpropagation and parameter iteration on the current value network by minimizing the mean square error loss between the current value network output value and the target value, thereby achieving an accurate evaluation of the long-term benefits of the task scheduling scheme in the cloud-edge collaborative environment. For the The i-th agent, whose Critic network is updated by minimizing the mean square error between the current Q-value estimate and the target Q-value; for the th agent... A smart agent Its current value network is updated by minimizing the mean square error between the current Q-value estimate and the target Q-value; Value loss function The calculation expression is: In the formula, For expectation operator, This is the global state. This refers to the global state at the next moment after the action is performed. To schedule the actions of the decision-making agent in the current state, To coordinate the actions of the resource-coordinating agent in the current state, For the first The instant reward obtained by each intelligent agent. For experience replay pool, Indicates the first The current value network output of each agent The target value; Target value Given the target value network, the calculation expression is: In the formula, As a discount factor, and These represent the target actions generated by the two target policy Actor networks at the next time step. Indicates the first The target value network output of each intelligent agent.

[0021] S5.3.2. Based on gradient information, the deterministic policy gradient is used to update the current policy network; After obtaining an effective value assessment, the system further utilizes the gradient information provided by the target value network to update the parameters of the two target policy Actor networks. For the , Each agent updates its current policy network according to a deterministic policy gradient. The expression is: In the formula, To average the global states sampled from the experience replay pool, Output pairs for policy network Jacobian matrix of network parameters For the first k The current policy network parameters of each agent For the first k Local observations of an agent For the current value network output to the first k Actions of an agent gradient, This is the global state. For the first k The action output by the agent To schedule the actions of the decision-making agent in the current state, To coordinate the actions of the resource-coordinating agent in the current state; S5.3.3. Based on synchronizing the network parameters of the current value network, current policy network, target value network, and target policy using a soft update method, training is performed; For the For each agent, the expression for the soft update method is: In the formula, For the first k The target policy network parameters of each agent For the first k The current policy network parameters of each agent This is the soft update coefficient. For the first k The target value network parameters of each agent For the first k The current value network parameters of each agent.

[0022] S5.4 Based on the network parameters after completing a preset number of training rounds, by judging whether the training has converged, output the optimized scheduling decision agent parameters and resource coordination agent parameters to complete the cloud-edge collaborative task time-series association scheduling method under computing power preemption constraints. A dual-agent cloud-edge collaborative task scheduling system under computing power fluctuations, specifically comprising: The mathematical optimization modeling module is used to execute S1: Based on the cloud-edge collaborative task time-series correlation scheduling problem CP-CETTS under computing power preemption constraints, construct a CP-CETTS mathematical optimization model; the CP-CETTS mathematical optimization model includes an optimization objective, a set of manufacturing-computing serial coupling constraints, and edge node resource capacity constraints. The MDP modeling module is used to execute S2: based on the CP-CETTS mathematical optimization model, the scheduling process is modeled as a dual-agent Markov decision process (MDP), and a dual-agent MDP model is constructed; the dual agents include a scheduling decision agent and a resource coordination agent; The dual-agent collaborative architecture module is used to execute step S3: based on the dual-agent MDP model, a dual-agent collaborative architecture consisting of a scheduling decision agent and a resource coordination agent is constructed; wherein, a heterogeneous graph including process nodes, machine nodes, and server nodes is constructed for the scheduling decision agent, and a heterogeneous graph neural network (HGNN) is used to extract unified features from the nodes in the heterogeneous graph and the edge connections between nodes to generate state embedding vectors, which are used as inputs to the policy network of the scheduling decision agent; the resource coordination agent directly uses the background load of the edge server, link status, and task execution feedback as inputs to the policy network of the resource coordination agent. The joint strategy module is used to execute S4: Based on the dual-agent collaborative architecture, a joint strategy is designed; the joint strategy is specifically as follows: the candidate actions output by the scheduling decision agent need to be filtered by the action masking layer filtering mechanism, the filtered scheduling decision agent actions and resource coordination agent actions are combined into a joint action, and an immediate reward function is defined to guide the dual agents to cooperate in optimization. The training execution module is used to execute S5: Based on the dual-agent collaborative architecture and joint policy, a centralized training and decentralized execution system for dual-agent deep deterministic policy gradient (MADDPG) is constructed. Through decentralized execution in the interaction phase and centralized training in the update phase, the system solves the cloud-edge collaborative task time-series scheduling problem under computing power preemption constraints, and outputs the training-optimized scheduling decision agent parameters and resource coordination agent parameters.

[0023] Beneficial effects of this invention: This invention addresses the problem of sequential task scheduling in cloud-edge collaborative manufacturing under conditions of computing power preemption and communication link fluctuations. It constructs a collaborative optimization method based on a dual-agent DDPG (Deep Deterministic Policy Gradient) approach, effectively separating manufacturing scheduling decisions from computing resource coordination and control. By introducing a scheduling decision agent and a resource coordination agent, this invention can jointly optimize the allocation of manufacturing, edge, and cloud resources while satisfying constraints such as process sequence, equipment compatibility, and the requirement that computing tasks can only be triggered after manufacturing is completed. Simultaneously, leveraging the representation capabilities of HGNN (Heterogeneous Graph Neural Network) for complex relationships between processes, machines, and servers, the system can more accurately extract workshop state features, improving its ability to perceive and model multi-source heterogeneous information. Furthermore, by introducing a Masking Layer, illegal actions that violate process logic and resource constraints are filtered from the decision output, reducing the invalid search space and improving policy training efficiency and scheduling feasibility. Compared to traditional single-agent or static rule scheduling methods, this invention can more effectively cope with the uncertainties caused by fluctuations in edge computing power and changes in link latency, significantly improve the real-time performance, stability and robustness of task scheduling, shorten the maximum system completion time, improve the efficiency of cloud-edge resource utilization, and enhance the engineering applicability of scheduling methods in complex industrial environments. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating the steps of a dual-agent cloud-edge collaborative task scheduling method under computing power fluctuations according to the present invention. Figure 2 This is an architecture diagram of the dual-agent cloud-edge collaborative task scheduling method under computing power fluctuations according to the present invention; Figure 3 This is the training diagram of the centralized training and distributed execution network of the present invention; Figure 4 This is a diagram illustrating the state transition process of the present invention; Figure 5 This is a training curve convergence plot of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] This invention employs a dual-agent cloud-edge collaborative task scheduling system under fluctuating computing power conditions to address the collaborative scheduling problem arising from the deep coupling between manufacturing processes and related computing tasks in environments characterized by random computing power preemption and fluctuating communication links. The system constructs a dual-agent collaborative architecture consisting of a scheduling decision-making agent and a resource coordination agent. The scheduling decision-making agent is responsible for the joint scheduling output during manufacturing execution, while the resource coordination agent is responsible for perceiving and controlling the dynamic state of the computing environment. By separating the "scheduling decision" and "resource coordination" functions, the system can achieve collaborative optimization of processing resources, computing resources, and environmental disturbances under the condition that manufacturing tasks and computing tasks are serially associated. Its overall architecture is as follows: The main function of the scheduling decision-making agent is to jointly decide on manufacturing process selection, processing equipment allocation, and subsequent computational task unloading targets. This agent uses HGNN (Heterogeneous Graph Neural Network) to uniformly model and extract features from process nodes, machine nodes, and cloud-edge server nodes in the workshop, mapping process requirements, equipment status, server performance, and their relationships into state representations that can be used for policy learning. Based on this, and combined with the DDPG method, the scheduling decision-making agent outputs joint action results for the current scheduling state, used to determine the processing equipment corresponding to the process to be executed and the pre-assigned nodes for associated computational tasks after the manufacturing task is completed. To ensure that the output actions meet the process and resource constraints of the industrial site, this invention sets a Masking Layer at the action output end of the scheduling decision-making agent. This masking layer filters candidate actions that do not meet the constraints based on preset process rules, equipment compatibility, server reachability, and communication coverage, thereby reducing the generation of illegal actions and improving the feasibility and security of the scheduling process.

[0027] The primary function of the resource coordination agent is to perceive dynamic disturbances in the computing environment and provide adjustment information for the scheduling strategy. Under the manufacturing-computing serial constraint, computing tasks are typically triggered after all manufacturing processes are completed, and their execution performance is directly affected by changes in edge server load and cloud-edge link quality. To address the potential for random occupation of edge computing resources by non-manufacturing background tasks and data transmission latency fluctuations caused by link fading, the resource coordination agent continuously monitors the edge server background load, link status, and computing task execution feedback, and generates uncertainty assessment results based on changes in environmental conditions. On this basis, the agent guides the scheduling decision-making agent by adjusting reward weights, policy preferences, or risk indications. When the system detects edge-side computing power strain or deteriorating communication conditions, the resource coordination agent can prompt the scheduling strategy to reduce its preference for high-risk nodes and prioritize computing nodes with relatively stable resources or better communication conditions, thereby improving the stability of the scheduling results under disturbed environments.

[0028] The technical solution proposed in this invention employs a dual-agent collaborative approach, dividing the complex manufacturing-computation serial scheduling process into two mutually cooperating parts: scheduling execution and environment coordination. This reduces the decision-making pressure on a single agent under high-dimensional states and multiple constraints. Compared to traditional single-agent or multi-agent parallel allocation methods, this scheme can more clearly define functional boundaries, reducing training difficulties and decision delays caused by excessively large action spaces or insufficient multi-agent collaboration. Furthermore, by combining the heterogeneous relationship modeling capabilities of HGNN, the continuous decision-making capabilities of DDPG, and the constraint filtering mechanism of the action occlusion layer, the system can optimize task completion time under conditions of fluctuating computing power and unstable links.

[0029] like Figure 1 As shown, a dual-agent cloud-edge collaborative task scheduling method under computing power fluctuations specifically includes the following steps: S1. Based on the cloud-edge collaborative task time-series correlation scheduling problem CP-CETTS under the defined computing power preemption constraint, construct a mathematical optimization model for CP-CETTS; The CP-CETTS mathematical optimization model includes: optimization objective, manufacturing-computation serial coupling constraint set, and edge node resource capacity constraint term; The cloud-edge collaborative task time-series correlation scheduling problem (CP-CETTS) under computing power preemption constraints can be defined as: There are One workpiece Need to Available machines The processing is completed on the above. Each workpiece After processing is completed, corresponding computational tasks will be generated. For each process Only one machine can be selected from its set of available machines for processing, and the processing times of the machines in the set vary. Under the premise of satisfying various constraints such as priority constraints and capacity constraints, the optimization objective of this invention is to minimize the maximum completion time. .

[0030] S1.1, The optimization objective is to minimize the maximum completion time, expressed as follows: (1); In the formula, Indicates the maximum completion time.

[0031] S1.2. Based on the optimization objective, define the set of manufacturing-computation serial coupling constraints; The manufacturing-computation serial coupling constraint set includes 5 manufacturing stage constraints and 10 computational unloading and dynamic disturbance constraints.

[0032] S1.2.1 Obtain manufacturing stage constraints, including: process completion time constraints, process sequence constraints, machine unique allocation constraints, machine processing non-overlapping constraints, and define the workpiece manufacturing completion time; (1) Process completion time constraint, the expression is as follows: (2); In the formula, Indicate process Completion time, Indicate process The start time, Indicates workpiece The One process, Indicates when the process Assigned to machine ,but ;otherwise , This represents the m-th machine. Indicate process In the machine The processing start time on the surface Indicate process A collection of compatible machines Indicates machine index; (2) Operation sequence constraint, used to ensure the order of operations within the same workpiece, is expressed as follows: (3); (3) Machine unique allocation constraint, used to ensure that each process is assigned to only one compatible machine, is expressed as follows: (4); (4) Non-overlapping machine processing constraint, used to ensure that the processing operations on the same machine do not overlap, is expressed as follows: (5); In the formula, Represents a sufficiently large positive number. Indicates the workpiece index. Indicates the process index. Indicates workpiece The The completion time of each process, Indicates workpiece The The start time of each process, Represented as 0-1 variables; (5) Define the workpiece manufacturing completion time, as shown in the following expression: (6); In the formula, Indicates workpiece Manufacturing completion time, Indicates workpiece The The completion time of each process, Indicates workpiece The number of processes, Represents the first in the set of workpieces One workpiece.

[0033] S1.2.2 Obtain computation unloading and dynamic disturbance constraints, including: unique allocation constraint for computation tasks, definition of time-varying available resources on the edge side, definition of dynamic link rate, definition of upload latency on the edge side, definition of upload latency on the cloud side, edge task start condition constraint, cloud task start condition constraint, definition of computation completion time on the edge side, definition of computation completion time on the cloud side, and definition of final completion time of the workpiece. (1) Unique assignment constraint for computation tasks, used to ensure that each computation task selects only one computation node, expressed as follows: (7); In the formula, Indicates workpiece The set of optional computing nodes corresponding to the computing task. Indicates the compute node index. Indicates when the workpiece computational tasks Unloaded to node ,but ;otherwise ; (2) Define the time-varying available resources on the edge side, as shown in the following expression: (8); In the formula, Indicates time, Indicates time The actual available computing power Indicates time The actual available memory, Indicates time The computing power that was forcibly seized Indicates time The memory that was forcibly occupied Represents edge server Total computing power Represents edge server Total memory, Represents the edge server index; (3) Define the dynamic link rate, as shown in the following expression: (9); In the formula, Indicates the actual effective transmission rate. Indicates the average effective transmission rate. This indicates the amount of fluctuation in transmission rate; (4) Define the upload latency on the edge side as follows: (10); In the formula, This represents the transmission latency from the manufacturing completion machine to the edge server for the computational task corresponding to the workpiece. Represents the first in the set of workpieces One workpiece, Indicates workpiece The amount of input data corresponding to the computation task. Indicates time Below, the effective transmission rate from the machine to the edge server; (5) Define the cloud-side upload latency as follows: (11); In the formula, Indicates workpiece The total transmission latency of the corresponding computing task from the edge server to the cloud server. This indicates the effective transmission rate from the edge server to the cloud server. Indicates workpiece Manufacturing completion time; (6) Edge-side task start condition constraints, the expression is as follows: (12); In the formula, Indicates the task is on the compute node At the beginning of Indicates workpiece The transmission latency of the corresponding computing task uploaded to the computing node; (7) Cloud-side task start condition constraints, the expression is as follows: (13); In the formula, Indicates workpiece The corresponding computing task begins execution on the edge server (referring to the moment when the edge side begins collaborative processing with the cloud). This represents the ready time function of the edge server. Indicates workpiece The corresponding computational task transmission latency from the manufactured machine to the edge server; (8) Define the completion time of edge side calculation, as shown in the following expression: (14); In the formula, Indicates workpiece The corresponding computation completion time of the computing task on the edge server. Indicates workpiece The corresponding start time of the computing task on the edge server. Indicates the actual available computing power. Indicates workpiece The computational workload of the corresponding computational task; (9) Define the cloud-side computation completion time as follows: (15); In the formula, This indicates the completion time of the computation task corresponding to the workpiece on the cloud server. Indicates workpiece The total transmission latency of the corresponding computing task from the edge server to the cloud server. This indicates the computing power of the cloud server. Indicates workpiece The amount of data returned by the corresponding computation task. This indicates the effective transmission rate from the cloud server to the edge server. (10) Define the final completion time of the workpiece, as follows: (16); In the formula, Indicates the final completion time of the workpiece. Represents a set of edge servers. Indicates the edge server index. Indicates workpiece The corresponding computation completion time of the computing task on the edge server. This indicates the completion time of the computation task corresponding to the workpiece on the cloud server. This represents the decision variable for edge server execution. This represents the decision-making variable for cloud server execution. S1.3. Define edge node resource capacity constraints based on the manufacturing-computing serial coupling constraint set; (17); In the formula, This indicates the CPU computing power required to execute the computational task corresponding to the workpiece. This indicates the memory requirements for executing the computational task corresponding to the workpiece. Indicates the actual available computing power. Indicates the actual available memory. Indicates workpiece The start time of the corresponding computing task on the edge server.

[0034] In this embodiment, Equation (1) is the optimization objective function used to minimize the maximum completion time of the system; Equations (2) to (6) describe the manufacturing stage constraints, wherein Equation (2) defines the process completion time, Equation (3) ensures the sequence of processes within the same workpiece, Equation (4) ensures that each process is assigned to only one compatible machine, Equation (5) ensures that the processing of processes on the same machine does not overlap, and Equation (6) defines the workpiece manufacturing completion time. Equations (7)–(17) describe computational unloading and dynamic disturbance constraints. Equation (7) ensures that each computational task selects only one computational node. Equations (8) and (9) define the time-varying available resources and dynamic link rate on the edge side, respectively. Equations (10) and (11) give the upload latency on the edge side and the cloud side. Equations (12) and (13) ensure that the computational task can only be executed after manufacturing is completed, upload is finished, and the node is available. Equations (14) and (15) define the completion time on the edge side and the cloud side, respectively. Equation (16) gives the final completion time of the workpiece. Equation (17) ensures that the resource constraints of the edge node are satisfied.

[0035] like Figure 2 As shown in the diagram, the architecture of the dual-agent cloud-edge collaborative task scheduling method under computing power fluctuations of the present invention specifically includes the following: S2. Based on the CP-CETTS mathematical optimization model, the scheduling process is modeled as a dual-agent Markov decision process (MDP), and a dual-agent MDP model is constructed. The dual-agent system includes: a scheduling decision-making agent and a resource coordination agent; This invention discloses a cloud-edge collaborative manufacturing-computing serial task scheduling method based on a dual-agent MADDPG in a computing power preemption environment. The collaborative optimization process of manufacturing tasks and their associated computing tasks is modeled as a dual-agent Markov decision process (MDP / Markov Game). This process mainly includes five parts: environment, state, action, reward, and state transition. Unlike traditional scheduling models that only consider manufacturing resources, this invention simultaneously considers factors such as manufacturing process steps, computing task triggering, random preemption of edge resources, and dynamic fluctuations in the cloud-edge link. Based on this, a dual-agent collaborative architecture consisting of a scheduling decision agent and a resource coordination agent is constructed to achieve joint optimization of manufacturing resources, computing resources, and environmental disturbances. Specifically, the method includes the following steps: S2.1 defines a manufacturing-computing serial scheduling environment based on Markov Decision Process (MDP). In the dual-agent Markov decision process (MDP) of this invention, the environment is the core carrier for agent interaction and decision execution. First, the environment needs to define the local observation space and action space of the scheduling decision agent and the resource coordination agent. Second, based on the Markov property, the environment completes transitions from the current state according to the state transition probability. To the next state The dynamic evolution of the action and the output of instant rewards for combined actions. This guides the dual agents to continuously learn and optimize strategies.

[0036] S2.2 Based on the manufacturing-computing serial scheduling environment, construct the state characteristics of the scheduling decision-making agent and the resource coordination agent; In the dual-agent cooperative scheduling system of this invention, the state is used to characterize the real-time operational features of the current manufacturing-computing system. Unlike traditional multi-agent scheduling methods that only describe the state of processes, machines, or transportation resources, the state in this invention includes not only the operational characteristics of manufacturing processes and equipment, but also information on computing-side server resources, link quality, and dynamic disturbances. During the complete scheduling process, the system maintains a state at every moment... The global state is denoted as .in, This represents the state characteristics of the scheduling decision-making agent. This represents the state characteristics of the resource coordination agent.

[0037] S2.2.1 Based on the manufacturing-computing serial scheduling environment, set the state characteristics of the scheduling agent; In this invention The state characteristics of the scheduling decision-making agent are represented by six local feature information items: (1) Indicates in Check whether all workpieces have been processed at any given time; (2) Indicates in Current process Are they eligible to be dispatched? (3) Indicates whether the workpiece has triggered a calculation task; (4) Indicates machine At any moment The earliest time that can be allocated; (5) Indicates machine In the interval The proportion of busyness within; (6) This indicates the average processing time for the current process across all compatible machines.

[0038] The details are shown in Table 1: Table 1: State characteristics of the scheduling agent S2.2.2 Based on the manufacturing-computing serial scheduling environment, set the state characteristics of the resource coordination agent; In this invention, The state characteristics of the resource coordination agent are represented, specifically including five local feature information items: (1) Represents edge server At any moment Available computing power equals total computing power minus preempted computing power; (2) This represents the total memory minus the memory that was preempted. (3) Indicates time Computing power being taken over by background tasks; (4) Indicates time Memory occupied by background tasks; (5) Represents computational task At any moment The execution feedback information includes waiting, execution completion, or delay status.

[0039] The details are shown in Table 2: Table 2: State characteristics and calculation formulas of resource coordination agents S2.3 Based on the manufacturing-computing serial scheduling environment, construct the action space of the scheduling decision-making agent, the action space of the resource coordination agent, and joint actions; The agent actions are used to complete the joint control of the pre-allocated manufacturing execution and computing tasks. The agent actions are represented as follows: ,in, Indicates the action to be selected for the process to be executed. Indicates the allocation of actions by the processing equipment. This indicates the pre-assigned node action for the corresponding computation task.

[0040] The resource coordination agent action is used to output environmental risk adjustment information, and the resource coordination agent action is represented as follows: ,in, Indicates the risk penalty weight. Indicates the delay penalty weight. This represents a risk indication vector used to guide the scheduling decision-making agent to avoid high-risk nodes.

[0041] The joint action is expressed as .

[0042] In the dual-agent cooperative scheduling system of this invention, the two agents each have independent but cooperative action spaces. The actions of the scheduling decision agent are mainly used to complete the joint control of the pre-allocation of manufacturing execution and computing tasks, and its actions can be represented as follows: ,in, Indicates the action to be selected for the process to be executed. Indicates the allocation of actions by the processing equipment. This represents the pre-allocated node action for the corresponding computational task. The actions of the resource coordination agent are mainly used to output environmental risk adjustment information, and their actions can be represented as follows: ,in, Indicates the risk penalty weight. Indicates the delay penalty weight. This represents a risk indication vector used to guide the scheduling decision-making agent to avoid high-risk nodes. The system at time [time value missing] The combined action can be represented as .

[0043] S2.4. Based on the optimization objective, define the reward functions for the scheduling decision agent and the resource coordination agent; In this dual-agent cooperative scheduling system, the two agents share a global optimization objective: minimizing the system's maximum completion time while simultaneously reducing the additional costs caused by edge resource preemption and link fluctuations. Therefore, at time [time value missing], The instant reward function is defined as: in, This indicates the change in upload transmission latency. This indicates changes in queuing delay. This indicates the cost of risk exposure caused by edge preemption and link fluctuations. This represents the coefficient of variation in upload transmission latency. This represents the coefficient of variation in queuing delay. This represents the risk exposure cost coefficient. Represents the global state. Indicates the state at the next moment. Indicates the maximum completion time.

[0044] S2.5. Based on the manufacturing-computing serial scheduling environment, the action space of the scheduling decision agent, the action space of the resource coordination agent, and joint actions, define the state transition process of the MDP and complete the construction of the dual-agent MDP model. The state transition process is specifically the temporal evolution process from the current state to the next state after the scheduling decision agent and the resource coordination agent in the manufacturing-computing serial scheduling environment collaboratively generate joint actions.

[0045] like Figure 4 As shown, the state transition represents the transition from the current state in a manufacturing-computing serial scheduling environment driven by joint decision-making between two agents. Next state The temporal evolution process. Specifically, at each decision step, the scheduling decision agent and the resource coordination agent first collaboratively generate joint actions. Subsequently, the environment executes the corresponding process processing and computation task scheduling operations based on process sequence constraints, machine processing non-overlapping constraints, edge-side task start condition constraints, cloud-side task start condition constraints, and edge node resource capacity constraints, thus advancing the system to the next discrete decision moment. During this process, process processing progress, earliest available machine time, workpiece manufacturing completion status, available computing power of edge nodes, effective link transmission rate, and task queue status are all updated. The system recalculates the state characteristics based on the updated global information and generates the state vector for the next moment. The aforementioned closed-loop process of perception-decision-evolution continues until all workpieces have completed manufacturing and associated computational tasks, at which point the scheduling process ends.

[0046] S3. Based on the dual-agent MDP model, construct a dual-agent collaborative architecture consisting of a scheduling decision agent and a resource coordination agent; The dual-agent collaborative architecture is as follows: a heterogeneous graph containing process nodes, machine nodes, and server nodes is constructed for the scheduling decision agent, and a heterogeneous graph neural network (HGNN) is used to extract unified features from the nodes in the heterogeneous graph and the edge connections between nodes to generate state embedding vectors, which serve as inputs to the policy network of the scheduling decision agent; the resource coordination agent also directly uses the background load of the edge server, link status, and task execution feedback as inputs to the policy network of the resource coordination agent. The server nodes include edge server nodes and cloud server nodes; S4. Design a joint strategy based on a dual-agent cooperative architecture; The joint strategy is as follows: the candidate actions output by the scheduling decision agent need to be filtered by the Action MaskingLayer mechanism, and the filtered actions of the scheduling decision agent and the actions of the resource coordination agent are combined into a joint action, and an immediate reward function is defined. S5. Based on the dual-agent collaborative architecture and joint strategy, a centralized training and decentralized execution system for dual-agent MADDPG (Multi-Agent Deep Deterministic Policy Gradient) is constructed. Through decentralized execution in the execution interaction phase and centralized training in the update phase, the problem of time-series correlation scheduling of cloud-edge collaborative tasks under computing power preemption is solved, and the method and system for time-series correlation scheduling of cloud-edge collaborative tasks under computing power preemption are completed. like Figure 3 The diagram shown is a training graph of the centralized training and distributed execution network of this invention. S5.1 Based on a dual-agent collaborative architecture, construct the current policy network, target policy network, current value network, and target value network for the scheduling decision agent and the resource coordination agent, respectively; This invention constructs a current policy Actor network for both the scheduling decision-making agent and the resource coordination agent. Target-Policy Actor Network Current Value Critic Network and the target value Critic network ; S5.2 Interaction Phase: Based on the current policy Actor network, through distributed execution and experience collection, the interaction experience tuples are stored in the experience replay pool; During the interaction phase, the scheduling decision-making agent and the resource coordination agent respectively, based on the current time... Local observation state of the scheduling decision-making agent Local observation state of resource coordination agent Generate actions independently; The interactive experience tuple Includes: current state Next state Joint actions and instant rewards ; During the interaction phase, the system operates in a distributed execution mode. The scheduling decision-making agent and the resource coordination agent each act according to the current time... Local observation status and Independent action generation. The scheduling decision agent uses a state embedding derived from a unified model of the heterogeneous relationship between "process-machine-server" using HGNN, combined with the current set of schedulable processes, machine states, and candidate computing node states, and then utilizes the current policy network of the scheduling decision agent. Output joint scheduling actions Meanwhile, the resource coordination agent, based on the edge server's background load, link status, and task execution feedback, uses the current policy network of the resource coordination agent. Output coordinated actions, the system at all times The combined action can be represented as To ensure that joint actions meet industrial mechanism constraints, candidate actions output by the scheduling decision-making agent must be processed by a Masking Layer. This masking mechanism filters candidate actions that do not meet the constraints based on process sequence, equipment compatibility, manufacturing completion triggering, server reachability, and edge resource constraints, reducing illegal actions at the source and improving scheduling feasibility and execution security. The masked joint actions are then processed. This is applied to the environment, driving the co-evolution of the manufacturing and computing environments. On the manufacturing side, processing steps are executed and the workpiece completion status is updated. On the computing side, after all manufacturing steps for the corresponding workpiece are completed, computing tasks are uploaded and executed, and the completion time of the computing tasks is updated based on the available resources of edge nodes, link speeds, and node queue status. Subsequently, the environment changes from its current state... Transition to the next state Simultaneously provide instant rewards Ultimately, the system will generate a complete interactive experience tuple for the current step. Store in the experience replay pool .

[0047] S5.3 Update Phase: Based on the experience replay pool, the current value network, current policy network, target value network, and target policy network are updated through centralized training; The target value network takes global state and joint actions as input; During the update phase, the system is in a centralized training mode. Unlike the interaction phase, which relies only on local observations, centralized training allows the Critic to access both the global state and joint actions simultaneously. This mitigates the non-stationarity issues caused by the simultaneous changes in the policies of other agents in a multi-agent environment and improves the stability of dual-agent collaborative learning.

[0048] Specifically, the system constructs the current Actor network for the scheduling decision-making agent and the resource coordination agent, respectively. Target Actor Network Current Critic network and the target Critic network In this context, both objective value Critic networks operate with a global state. With joint actions As input, the long-term returns of joint decision-making are evaluated.

[0049] S5.3.1 Based on the experience replay pool, the current value network and gradient information are updated by calculating the target Q-value and the value loss function; The system extracts historical interaction samples from the experience replay pool, constructs target values ​​using the target policy network and the target value network, and performs gradient backpropagation and parameter iteration on the current value network by minimizing the mean square error loss between the current value network output value and the target value, thereby achieving an accurate evaluation of the long-term benefits of the task scheduling scheme in the cloud-edge collaborative environment. For the An agent whose Critic network is updated by minimizing the mean square error between the current Q-value estimate and the target Q-value.

[0050] The expression for calculating the value Critic loss function is as follows: In the formula, For expectation operator, This is the global state. This refers to the global state at the next moment after the action is performed. To schedule the actions of the decision-making agent in the current state, To coordinate the actions of the resource-coordinating agent in the current state, For the first The instant reward obtained by each intelligent agent. For experience replay pool, Indicates the first The current value network output of each agent The target value; Target value Given the target value network, the calculation expression is: In the formula, As a discount factor, and These represent the target actions generated by the two target policy Actor networks at the next time step. Indicates the first The target value network output of each intelligent agent.

[0051] S5.3.2. Based on gradient information, the deterministic policy gradient is used to update the current policy network; After obtaining an effective value assessment, the system further utilizes the gradient information provided by the target value Critic network to update the parameters of the two target policy Actor networks. For the ... There are *n* agents, whose policy updates follow a deterministic policy gradient, expressed as: In the formula, Output pairs for policy network Jacobian matrix of network parameters For the first k Local observations of an agent For the current value network output to the first k Actions of an agent gradient, For the first k The action output by the intelligent agent; Among them, the Actor network of the scheduling decision-making agent is responsible for learning the joint output strategy of "process selection - machine allocation - computational task pre-unloading"; the Actor network of the resource coordination agent is responsible for learning the dynamic adjustment strategy of risk preference, latency penalty and node stability preference, and through risk guidance vectors. Alternatively, reward adjustment parameters can guide the scheduling decision-making agent. To suppress drastic fluctuations in network parameters and improve training stability, this paper further employs a soft update method to synchronize the current network and target network parameters.

[0052] S5.3.3. Based on synchronizing the parameters of the current value network, current policy network, target value network, and target policy network using a soft update method, training is performed; For the For each agent, the expression for the soft update method is: In the formula, For the first k The target policy network parameters of each agent For the first k The current policy network parameters of each agent This is the soft update coefficient. For the first k The target value network parameters of each agent For the first k The current value network parameters of each agent This is the soft update coefficient, typically much smaller than 1. By slowly tracking and updating the Actor and Critic target networks, training oscillations can be effectively reduced, enhancing the convergence stability of the two agents under complex computing power fluctuations.

[0053] S5.4 Based on the network parameters after completing a preset number of training rounds, by judging whether the training has converged, output the optimized scheduling decision agent parameters and resource coordination agent parameters to complete the cloud-edge collaborative task time-series association scheduling method under computing power preemption constraints. Figure 5 The training curve is shown on a 10×6×4×1 instance. The pseudocode for the specific training steps is shown in Algorithm 1: It should be noted that the above are merely preferred embodiments of this application and do not limit the scope of patent protection of this application. Any equivalent structural or procedural changes made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of this application.

Claims

1. A method for task scheduling in a double-agent cloud-edge collaborative system under fluctuating computing power, characterized in that, Includes the following steps: S1. Based on the cloud-edge collaborative task time-series correlation scheduling problem CP-CETTS under the defined computing power preemption constraint, construct a mathematical optimization model for CP-CETTS; The CP-CETTS mathematical optimization model includes: optimization objective, manufacturing-computation serial coupling constraint set, and edge node resource capacity constraint term; S2. Based on the CP-CETTS mathematical optimization model, the scheduling process is modeled as a dual-agent Markov decision process (MDP), and a dual-agent MDP model is constructed. The dual-agent system includes: a scheduling decision-making agent and a resource coordination agent; S3. Based on the dual-agent MDP model, construct a dual-agent collaborative architecture consisting of a scheduling decision agent and a resource coordination agent; The dual-agent collaborative architecture is as follows: a heterogeneous graph containing process nodes, machine nodes, and server nodes is constructed for the scheduling decision agent, and a heterogeneous graph neural network (HGNN) is used to extract unified features from the nodes in the heterogeneous graph and the edge connections between nodes to generate state embedding vectors, which serve as inputs to the policy network of the scheduling decision agent; the resource coordination agent also directly uses the background load of the edge server, link status, and task execution feedback as inputs to the policy network of the resource coordination agent. The server nodes include edge server nodes and cloud server nodes; S4. Design a joint strategy based on a dual-agent cooperative architecture; The joint strategy is as follows: the candidate actions output by the scheduling decision agent need to be filtered by the Action MaskingLayer filtering mechanism, and the filtered scheduling decision agent actions and resource coordination agent actions are combined into a joint action. An immediate reward function is defined to guide the two agents to cooperate in optimization. S5. Based on the dual-agent collaborative architecture and joint strategy, a centralized training and decentralized execution system for the dual-agent deep deterministic policy gradient (MADDPG) is constructed. By decentralizing execution during the execution interaction phase and centralizing training during the update phase, the problem of time-series correlation scheduling of cloud-edge collaborative tasks under computing power preemption is solved, thus completing the time-series correlation scheduling method for cloud-edge collaborative tasks under computing power preemption.

2. The method according to claim 1, wherein, S1 specifically includes the following steps: S1.1 The optimization objective is to minimize the maximum completion time; S1.

2. Based on the optimization objective, define the set of manufacturing-computation serial coupling constraints; The manufacturing-computation serial coupling constraint set includes manufacturing stage constraints as well as computational unloading and dynamic disturbance constraints; The manufacturing stage constraints specifically include: process completion time constraints, process sequence constraints, machine unique allocation constraints, non-overlapping machine processing constraints, and definition of workpiece manufacturing completion time. The computational offloading and dynamic disturbance constraints specifically include: unique allocation constraints for computational tasks, definition of time-varying available resources on the edge side, definition of dynamic link rate, definition of upload latency on the edge side, definition of upload latency on the cloud side, edge-side task start condition constraints, cloud-side task start condition constraints, definition of edge-side computation completion time, definition of cloud-side computation completion time, and definition of final completion time of the workpiece. S1.

3. Define edge node resource capacity constraints based on the manufacturing-computing serial coupling constraint set; The expression for the edge node resource capacity constraint is: In the formula, represents the CPU computing power requirement required by the workpiece corresponding computing task for execution, represents the memory requirement required by the workpiece corresponding computing task for execution, represents the actual available computing power, represents the actual available memory, represents the starting execution time of the workpiece corresponding computing task on the edge server.

3. The method of claim 1, wherein, S2 specifically includes the following steps: S2.

1. Based on Markov Decision Process (MDP), define a manufacturing-computing serial scheduling environment; Specifically, in a manufacturing-computing serial scheduling environment, the system records the global state at time as wherein, represents the state feature of the scheduling decision agent, represents the state feature of the resource coordination agent; S2.2 Based on the manufacturing-computing serial scheduling environment, construct the state characteristics of the scheduling decision-making agent and the resource coordination agent; S2.3 Based on the manufacturing-computing serial scheduling environment, construct the action space of the scheduling decision-making agent, the action space of the resource coordination agent, and joint actions; The agent action is used for completing the joint control of pre-allocated manufacturing execution and computing tasks, and the agent action is represented as wherein, represents a process selection action to be executed, represents a processing equipment allocation action, represents a pre-allocated node action corresponding to a computing task; The resource coordination agent action is used for outputting environment risk adjustment information, and the resource coordination agent action is represented as wherein, represents a risk penalty weight, represents a time delay penalty weight, represents a risk indication vector for guiding the scheduling decision agent to avoid high-risk nodes. The joint action is expressed as ; S2.

4. Based on the optimization objective, define the immediate reward functions for the scheduling decision agent and the resource coordination agent; Instant reward function The expression is: in, Indicates the maximum completion time. Represents the global state. Indicates the state at the next moment. This indicates the change in upload transmission latency. This indicates changes in queuing delay. This indicates the cost of risk exposure caused by edge preemption and link fluctuations. This represents the coefficient of variation in upload transmission latency. This represents the coefficient of variation in queuing delay. This represents the risk exposure cost coefficient; S2.

5. Based on the manufacturing-computing serial scheduling environment, the action space of the scheduling decision agent, the action space of the resource coordination agent, and joint actions, define the state transition process of the MDP and complete the construction of the dual-agent MDP model. The state transition process is specifically the temporal evolution process from the current state to the next state after the scheduling decision agent and the resource coordination agent in the manufacturing-computing serial scheduling environment collaboratively generate joint actions.

4. The dual-agent cloud-edge collaborative task scheduling method under computing power fluctuations according to claim 3, characterized in that, S2.2 specifically includes the following steps: S2.2.1 Based on the manufacturing-computing serial scheduling environment, set the state characteristics of the scheduling agent; The state characteristics of the scheduling agent specifically include six local feature information items: (1) Indicates in Whether all workpieces have been processed at any given time. Indicates time, Indicates workpiece The One process, Indicates the workpiece index. Indicates the process index; (2) Indicates in Current process Are they eligible to be dispatched? (3) Indicates whether the workpiece has triggered a calculation task; (4) Indicates machine At any moment The earliest time that can be allocated, express The first machine, Indicates machine index; (5) Indicates machine In the interval The proportion of busyness within; (6) This indicates the average processing time of the current process across all compatible machines; S2.2.2 Based on the manufacturing-computing serial scheduling environment, set the state characteristics of the resource coordination agent; The state characteristics of the resource coordination agent specifically include five local feature information items: (1) Indicates the edge server at time Available computing power equals total computing power minus preempted computing power; (2) This represents the total memory minus the memory that was preempted. (3) Indicates time Computing power being taken over by background tasks; (4) Indicates time Memory occupied by background tasks; (5) Indicates the computational task at time t. The execution feedback information includes waiting, execution completion, and delay status.

5. The dual-agent cloud-edge collaborative task scheduling method under computing power fluctuations according to claim 1, characterized in that, S5 specifically includes the following steps: S5.1 Based on a dual-agent collaborative architecture, construct the current policy network, target policy network, current value network, and target value network for the scheduling decision agent and the resource coordination agent, respectively; S5.2 Interaction Phase: Based on the current policy Actor network, through distributed execution and experience collection, the interaction experience tuples are stored in the experience replay pool; During the interaction phase, the scheduling decision-making agent and the resource coordination agent respectively, based on the current time... Local observation state of the scheduling decision-making agent Local observation state of resource coordination agent Generate actions independently; The interactive experience tuple includes: global state The state at the next moment Joint actions and instant rewards ; S5.3 Update Phase: Based on the experience replay pool, the current value network, current policy network, target value network, and target policy network are updated through centralized training; The target value network takes global state and joint actions as input; S5.4 Based on the network parameters after completing a preset number of training rounds, by judging whether the training has converged, output the optimized scheduling decision agent parameters and resource coordination agent parameters to complete the cloud-edge collaborative task time-series association scheduling method under the condition of limited computing power preemption.

6. The dual-agent cloud-edge collaborative task scheduling method under computing power fluctuations according to claim 5, characterized in that, S5.3 specifically includes the following steps: S5.3.1 Based on the experience replay pool, the current value network and gradient information are updated by calculating the target Q-value and the value loss function; Specifically, the system extracts historical interaction samples from the experience replay pool, constructs a target value using the target policy network and the target value network, and performs gradient backpropagation and parameter iteration on the current value network by minimizing the mean square error loss between the current value network output value and the target value, thereby achieving an accurate evaluation of the long-term benefits of the task scheduling scheme in the cloud-edge collaborative environment. For the The i-th agent, whose Critic network is updated by minimizing the mean square error between the current Q-value estimate and the target Q-value; for the th agent... A smart agent Its current value network is updated by minimizing the mean square error between the current Q-value estimate and the target Q-value; S5.3.

2. Based on gradient information, the deterministic policy gradient is used to update the current policy network; After obtaining an effective value assessment, the system further utilizes the gradient information provided by the target value network to update the parameters of the two target policy Actor networks. For the , Each agent updates its current policy network according to a deterministic policy gradient. The expression is: In the formula, To average the global states sampled from the experience replay pool, Output pairs for policy network The Jacobian matrix of network parameters, For the first k The current policy network parameters of each agent For the first k Local observations of an agent For the current value network output to the first k Actions of an agent gradient, This is the global state. For the first k The action output by the agent To schedule the actions of the decision-making agent in the current state, To coordinate the actions of the resource-coordinating agent in the current state; S5.3.

3. Based on synchronizing the network parameters of the current value network, current policy network, target value network, and target policy using a soft update method, training is performed; For the For each agent, the expression for the soft update method is: In the formula, For the first k The target policy network parameters of each agent This is the soft update coefficient. For the first k The current policy network parameters of each agent For the first k The target value network parameters of each agent For the first k The current value network parameters of each agent.

7. The dual-agent cloud-edge collaborative task scheduling method under computing power fluctuations according to claim 6, characterized in that, In S5.3.1, the value loss function The calculation expression is: In the formula, For expectation operator, This is the global state. To schedule the actions of the decision-making agent in the current state, To coordinate the actions of the resource-coordinating agent in the current state, For the first The instant reward obtained by each intelligent agent. This refers to the global state at the next moment after the action is performed. For experience replay pool, Indicates the first The current value network output of each agent The target value; Target value Given the target value network, the calculation expression is: In the formula, As a discount factor, and These represent the target actions generated by the two target policy Actor networks at the next time step. Indicates the first The target value network output of each intelligent agent.

Images