Distributed scheduling method and system for large-scale computing power network

Abstract

The invention discloses a distributed scheduling method and system for a large-scale computing power network, and the method comprises the steps: carrying out the offline training of a lightweight reinforcement learning agent through a multi-agent depth deterministic policy gradient algorithm, issuing the trained agent to a scheduling router, and deploying the scheduling router to a plurality of access points of the computing power network; on the basis of a computing power network and in combination with a dynamic routing gating architecture, independent scheduling decision making is carried out on local observation information through a lightweight reinforcement learning agent in a scheduling router, and a composite scheduling decision is obtained; and issuing the composite scheduling decision to a data plane of a scheduling router, and performing multi-path flow distribution to realize distributed scheduling of the computing power network. According to the embodiment of the invention, the overall health state of the network can be maintained while the end-to-end time delay of the task is optimized, and the real-time performance of the computing power network is improved. The method can be widely applied to the technical field of network scheduling.

Description

A Distributed Scheduling Method and System for Large-Scale Computing Networks Technical Field

[0001] This application relates to the field of network scheduling technology, and in particular to a distributed scheduling method and system for large-scale computing networks. Background Technology

[0002] With the rapid development of emerging technologies such as artificial intelligence and big data, the demand for computing power is exploding. As a new type of network architecture, computing power networks deeply integrate distributed computing resources with network resources, providing users with on-demand, flexible, and efficient computing services. Computing power network scheduling, as the core technology of computing power networks, aims to intelligently allocate users' computing tasks to appropriate computing nodes and plan efficient network transmission paths to optimize end-to-end performance.

[0003] However, with the continuous expansion of network scale and the increasing complexity of business needs, relevant computing power network scheduling methods face the following problems in practical applications. First, the computing power network scheduling problem itself has complex decision space characteristics. Scheduling decisions involve two levels: discrete selection of computing power nodes and continuous multi-path traffic allocation, forming a typical hybrid action space problem. Traditional deep reinforcement learning algorithms are usually designed for purely discrete or purely continuous action spaces respectively. When directly applied to hybrid action spaces, they often encounter algorithmic mismatches. Discretizing the continuous part leads to an exponential increase in the action space with the number of paths, thus causing the "curse of dimensionality." Modifying the algorithm to output both discrete and continuous actions simultaneously requires not only a deep transformation of the network structure but also a fundamental adjustment of the learning paradigm, making the algorithm implementation more complex and less prone to convergence.

[0004] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0005] The main objective of this application is to propose a distributed scheduling method and system for large-scale computing networks, which can optimize end-to-end task latency while maintaining the overall health of the network and improving the real-time performance of the computing network.

[0006] To achieve the above objectives, one aspect of this application proposes a distributed scheduling method for a large-scale computing network. The method includes: offline training of lightweight reinforcement learning agents using a multi-agent deep deterministic policy gradient algorithm, followed by distribution to a scheduling router, and deployment of the scheduling router to several access points of the computing network; based on the computing network and combined with a dynamic routing gating architecture, independent scheduling decisions are made by the lightweight reinforcement learning agents in the scheduling router based on local observation information to obtain a composite scheduling decision; the composite scheduling decision is then distributed to the data plane of the scheduling router for multi-path traffic allocation, thereby realizing distributed scheduling of the computing network.

[0007] In some embodiments, the step of offline training of lightweight reinforcement learning agents using a multi-agent deep deterministic policy gradient algorithm followed by deployment to a scheduling router, and then deploying the scheduling router to several access points of the computing power network, includes: offline training of the lightweight reinforcement learning agents using the multi-agent deep deterministic policy gradient algorithm; evaluating the joint actions of all lightweight reinforcement learning agents using a global critic network to obtain trained lightweight reinforcement learning agents; periodically deploying the trained lightweight reinforcement learning agents to the scheduling router, wherein the scheduling router includes a control plane and a data plane, the control plane is used to receive local information and make scheduling decisions, and the data plane is used to execute the scheduling decisions; and deploying the scheduling router based on several access nodes of the computing power network.

[0008] In some embodiments, the step of offline training of the lightweight reinforcement learning agent using the multi-agent deep deterministic policy gradient algorithm and evaluating the joint actions of all the lightweight reinforcement learning agents using a global critic network to obtain the trained lightweight reinforcement learning agent includes: offline training of the lightweight reinforcement learning agent using the multi-agent deep deterministic policy gradient algorithm, wherein the multi-agent deep deterministic policy gradient algorithm includes several actor networks and a global critic network, wherein each actor network corresponds to one lightweight reinforcement learning agent; acquiring the local observation information of the lightweight reinforcement learning agent and inputting it into the actor network to obtain the actions of the lightweight reinforcement learning agent; and acquiring the hidden state information of the lightweight reinforcement learning agent using an in-band network telemetry method. Global information is constructed by combining the actions and local state of the lightweight reinforcement learning agent and input into the global critic network to obtain the expected cumulative reward of the actions of the lightweight reinforcement learning agent. An experience replay pool is constructed by combining the actions and local state of the lightweight reinforcement learning agent, the expected cumulative reward, and the actions of the lightweight reinforcement learning agent at the next time step. Experience sample data is randomly extracted from the experience replay pool by the scheduling controller in the computing power network, and the global critic network is updated by minimizing the temporal difference error. The actor network is updated based on the expected cumulative reward. A global objective function is designed, and the lightweight reinforcement learning agent is trained based on the updated global critic network and the updated actor network to obtain the trained lightweight reinforcement learning agent.

[0009] In some embodiments, the global objective function is a weighted negative sum of the average end-to-end latency of the task and the maximum link utilization of the network. The average end-to-end latency of the task includes transmission latency and computation latency, and the maximum link utilization of the network represents the maximum value of the utilization of all links in the computing network. The specific expression of the global objective function is as follows: In the above formula, Indicates global reward. This represents the average end-to-end latency for all tasks. This represents the maximum link utilization of the network. and This represents the weighting coefficient, used to balance the two objectives of latency optimization and network health maintenance.

[0010] In some embodiments, the local observation information includes task requirement information, local link status information, and estimated computational latency of candidate computing nodes, wherein: the task requirement information includes the amount of data currently arriving at the task and the task feature vector; the local link status information represents the real-time status of each outgoing link of the access node; and the estimated computational latency of candidate computing nodes represents the estimated computational latency of the current task on each node in the candidate computing node set.

[0011] In some embodiments, the composite scheduling decision includes a discrete action space and a continuous action space, wherein: the discrete action space represents the target computing power node selected by the current task; and the continuous action space represents the multipath traffic allocation ratio to the target computing power node.

[0012] In some embodiments, the dynamic routing gating architecture includes a shared backbone network, a router module, and an expert network, wherein: the shared backbone network represents a multilayer perceptron, which takes into account the local observation information of the lightweight reinforcement learning agent, and extracts and outputs a high-dimensional state feature vector through several fully connected layers and nonlinear activation functions; the router module introduces a dynamic bias term based on usage frequency, maintains the corresponding centroid vector according to the candidate computing power nodes, performs dot product calculation in combination with the high-dimensional state feature vector, and selects the candidate computing power nodes based on the calculation result through the Gumbel-Softmax function; the expert network is bound to the candidate computing power nodes respectively, takes the high-dimensional state feature vector as input, outputs a dimension vector through fully connected layers, and obtains the multipath traffic allocation ratio of the target computing power node by normalizing the dimension vector.

[0013] In some embodiments, the expression for the Gumbel-Softmax function is as follows: In the above formula, , They represent experts respectively. , Affinity score, , Representing nodes respectively , Noise sampled from the Gumbel(0,1) distribution. Indicates temperature parameter, Represents the set of candidate computing power nodes. This indicates traversing the set of candidate computing power nodes. All nodes in This represents the soft selection probability after Gumbel-Softmax processing.

[0014] In some embodiments, the step of sending the composite scheduling decision to the data plane of the scheduling router for multi-path traffic allocation to achieve distributed scheduling of the computing power network includes: sending the composite scheduling decision to the data plane of the scheduling router; pre-calculating several paths from each access node to the computing power node using the k-shortest path algorithm; and allocating traffic to several of the paths according to the multi-path traffic allocation ratio of the target computing power node in the composite scheduling decision to achieve distributed scheduling of the computing power network.

[0015] To achieve the above objectives, another aspect of this application proposes a distributed scheduling system for a large-scale computing network. The system includes: a first module, used to train lightweight reinforcement learning agents offline using a multi-agent deep deterministic policy gradient algorithm and then distribute the training to a scheduling router, and deploy the scheduling router to several access points of the computing network; a second module, used to, based on the computing network and combined with a dynamic routing gating architecture, use the lightweight reinforcement learning agents in the scheduling router to make independent scheduling decisions on local observation information to obtain a composite scheduling decision; and a third module, used to distribute the composite scheduling decision to the data plane of the scheduling router for multi-path traffic allocation, thereby realizing distributed scheduling of the computing network.

[0016] The embodiments of this application include at least the following beneficial effects: This application provides a distributed scheduling method and system for large-scale computing networks. The scheme uses a multi-agent deep deterministic policy gradient algorithm to train lightweight reinforcement learning agents offline and then distributes them to the scheduling router. During the online execution phase, the agents on each scheduling router make decisions independently based entirely on local information without interacting with the controller or other agents, thereby achieving low-latency response. During the offline training phase, the network scheduling controller maintains a global learning system, using global information to guide the collaborative learning of all agents. Based on the computing network and combined with a dynamic routing gating architecture, lightweight reinforcement learning agents in the scheduling router independently make scheduling decisions based on local observation information, resulting in composite scheduling decisions. Through the dynamic routing gating mechanism, transmission latency is effectively reduced. It can keenly perceive the nonlinear impact of network congestion on transmission latency and make intelligent trade-offs between "relatively distant low-load nodes" and "congested near-end nodes," thereby optimizing overall transmission efficiency. Finally, the data plane of the composite scheduling decision is used for multi-path traffic distribution, avoiding the huge control overhead caused by distributing routing configurations to all nodes in the network in a centralized architecture. This eliminates the information collection and decision deployment delays of centralized scheduling, achieving sub-second scheduling response and improving the real-time performance of the computing network. Attached Figure Description

[0017] Figure 1 is a flowchart of a distributed scheduling method for a large-scale computing network provided in an embodiment of this application; Figure 2 is a structural diagram of a distributed scheduling system for a large-scale computing network provided in an embodiment of this application; Figure 3 is a schematic diagram of a specific implementation process of distributed scheduling provided in an embodiment of this application; Figure 4 is a schematic diagram of a specific implementation system of distributed scheduling provided in an embodiment of this application; Figure 5 is a schematic diagram of the actor network structure of a reinforcement learning agent provided in an embodiment of this application; Figure 6 is a schematic diagram of a training framework for a multi-agent deep deterministic policy gradient algorithm provided in an embodiment of this application; Figure 7 is a schematic diagram comparing the maximum link utilization of the network under different network topologies provided in an embodiment of this application; Figure 8 is a schematic diagram comparing the average transmission delay under different network topologies provided in an embodiment of this application; Figure 9 is a schematic diagram comparing the average end-to-end delay under different network topologies provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application.

[0019] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if” or “when” as used herein may be interpreted as “when…” or “in response to determination.”

[0020] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0022] Some shortcomings exist in related technologies, such as: 1) Severe delays in information collection and decision deployment. Most current scheduling schemes adopt a centralized framework, in which the scheduling decision process includes three main stages: global information collection, scheduling scheme calculation, and decision deployment to routers. Although centralized scheduling schemes can make accurate decisions based on comprehensive global information, the delay problem in their control loop cannot be ignored. Specifically, in the entire control loop, the centralized controller needs a lot of time to wait for data information from remote routers; after the decision scheme is calculated, the router needs to wait for the remote controller to issue the decision. Although machine learning-based scheduling algorithms are very fast in the model inference stage, as the algorithm's computation time shortens, the information collection and decision issuance stages become new bottlenecks. Especially with the expansion of network scale, the delays in these two stages show superlinear growth, making it difficult to meet the sub-second or even millisecond response requirements of real-time AI services; 2) Lack of joint optimization of computing resources and network resources. Most related scheduling methods focus on optimizing network resources while ignoring the coordinated scheduling of computing resources. The execution process of AI services involves two stages: transmission and computation. The end-to-end response latency is jointly determined by the latency of these two components. Computation latency is determined by the characteristics of the computing task and the performance of the computing nodes, while transmission latency depends not only on the selection of computing nodes but also on network path planning. Simply optimizing network or computing resources cannot achieve globally optimal performance; 3) It ignores the overall network health. Many scheduling algorithms are designed with a narrow focus on optimizing the end-to-end performance metrics of a single task or a batch of tasks, such as average response latency. This task-centric optimization model may come at the expense of the overall network health. For example, to minimize the latency of a certain task, the scheduler may continuously select the same optimal path, resulting in consistently high link utilization on that path, forming a network hotspot. This not only increases the risk of network congestion and reduces network robustness but also negatively impacts the performance of other services in the network.

[0023] In view of this, this application provides a distributed scheduling method for large-scale computing networks. It employs a "centralized training, distributed execution" scheduling architecture and combines a multi-agent deep reinforcement learning algorithm to jointly optimize computing and network resources. Lightweight reinforcement learning agents are deployed on the scheduling router to make independent scheduling decisions based on local state information. A multi-agent deterministic policy gradient algorithm with a global critic network is used for offline training to address the instability of distributed learning. A dynamic routing-gated actor network architecture is designed, using an expert network mechanism to handle the mixed action space and achieve collaborative optimization of computing node selection and traffic allocation. The maximum link utilization rate of the network is introduced as a penalty term in the reward function to maintain the overall health of the network while optimizing end-to-end task latency. This method eliminates the information collection and decision-making delays of centralized scheduling, achieving sub-second scheduling response and significantly improving the real-time performance, scalability, and robustness of the computing network, making it particularly suitable for latency-sensitive AI business scenarios.

[0024] Referring to Figure 1, which is a flowchart of a distributed scheduling method for a large-scale computing network provided by an embodiment of the present invention, the method includes the following steps: S100, after offline training of a lightweight reinforcement learning agent using a multi-agent deep deterministic policy gradient algorithm, the training is distributed to a scheduling router, and the scheduling router is deployed to several access points of the computing network; it should be noted that in some embodiments, step S100 may include S110 to S130.

[0025] S110. The lightweight reinforcement learning agent is trained offline using a multi-agent deep deterministic policy gradient algorithm. The joint action of all lightweight reinforcement learning agents is evaluated using a global critic network to obtain the trained lightweight reinforcement learning agent. Furthermore, it should be noted that step S110 may include S111 to S116.

[0026] In this embodiment, a multi-agent deep deterministic policy gradient algorithm is used for offline training on the computing network scheduling controller. A global critic network is used to evaluate the joint actions of all agents, guiding them to collaboratively optimize the global objective. This embodiment adopts a "centralized training, distributed execution" learning paradigm. During the online execution phase, agents on each scheduling router make decisions independently based entirely on local information, without interacting with the controller or other agents, thus achieving low-latency response. During the offline training phase, the computing network scheduling controller maintains a global learning system, using global information to guide the collaborative learning of all agents.

[0027] S111. Offline training of lightweight reinforcement learning agents is performed using the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm. The MADDPG algorithm includes several actor networks and a global critic network, with each actor network corresponding to a lightweight reinforcement learning agent. In this embodiment, as shown in Figure 6, the training system is based on the MADDPG algorithm. This algorithm is an actor-critic architecture, containing... A network of actors (actions) A global critic network .

[0028] S112. Obtain local observation information of the lightweight reinforcement learning agent and input it into the actor network to obtain the actions of the lightweight reinforcement learning agent; in this embodiment, each actor network This corresponds to an intelligent agent acting as a scheduling router, employing a dynamic routing gating architecture. During training, the actor network... According to the intelligent agent Local observations Output Action ,in The parameters represent the actor network.

[0029] S113. Hidden state information of the lightweight reinforcement learning agent is obtained through in-band network telemetry, and global information is constructed by combining the actions and local state of the lightweight reinforcement learning agent and input into the global critic network to obtain the expected cumulative reward of the actions of the joint lightweight reinforcement learning agent; in this embodiment, the global critic network It is a core component of the training phase, used to evaluate the value of the joint actions of all agents. Unlike the critic network in traditional single-agent RL, which only uses the state and actions of a single agent as input, the global critic network uses global information as input, including the local states of all agents. Actions of all intelligent agents And the hidden states that these agents cannot directly observe. Hidden state This includes network topology information, link utilization of intermediate conventional routers, and the overall network traffic forwarding path. This information can be obtained through in-band network telemetry (INT) technology. The output of the global critic network is the global Q-value, representing the state... Execute joint actions Expected cumulative rewards; ;in, These represent the parameters of the critic network. Indicates from time Initial cumulative discount rewards, This is the discount factor.

[0030] By introducing a global critic network, the actions of all agents become visible to each other, and some hidden states also become visible to all agents. This makes the environment stable from the perspective of any single agent, because the state transitions of the environment no longer depend on the unknown policies of other agents. Each agent can stably train towards a common global optimization goal.

[0031] S114. Combining the actions and local state of the lightweight reinforcement learning agent, the expected cumulative reward, and the actions of the lightweight reinforcement learning agent at the next time step, an experience replay pool is constructed. In this embodiment, the simulation environment module in the network scheduling controller constructs a high-fidelity computing network simulator to simulate the complete process of task arrival, scheduling decision, traffic transmission, and task computation. The training process adopts an experience replay mechanism, maintaining a replay buffer. Store historical interaction experience quadruple In each training iteration, the controller randomly samples a mini-batch of empirical samples from the replay buffer and then performs an update step.

[0032] S115. The scheduling controller in the computing power network randomly extracts experience sample data from the experience replay pool and updates the global critic network by minimizing the temporal difference (TD) error. In this embodiment, the global critic network is updated by minimizing the temporal difference (TD) error. The TD objective is defined as: In the above formula, Indicates the next state Below, each agent uses the actions generated by the target actor network. and Let represent the parameters of the target critic network and the target actor network, respectively. These parameters are slowly obtained from the online network parameters through a soft update mechanism. The loss function of the critic network is the mean squared TD error, and its expression is: The parameters of the critic network are updated using gradient descent, and the expression is as follows: ;in, is the learning rate of the critic network.

[0033] S116. Update the actor network based on the expected cumulative reward, design a global objective function, and train the lightweight reinforcement learning agent based on the updated global critic network and the updated actor network to obtain the trained lightweight reinforcement learning agent.

[0034] In this embodiment, the actor network is updated, and each agent... The actor network is updated by maximizing the Q-value evaluated by the global critic network. The policy gradient theorem gives the policy parameters. The gradient calculation formula is as follows: This gradient indicates that the actor network updates in a direction that maximizes the Q-value evaluation of the global critic network by maximizing the action it outputs. In practice, this is achieved by first fixing the actions of other agents. Then calculate when the agent Change its action Rate of change of Q value Finally, the gradient is backpropagated to the actor network parameters using the chain rule. .

[0035] The formula for updating actor network parameters is: In the above formula, is the learning rate of the actor network.

[0036] Furthermore, to improve training stability, the system maintains a target actor network and a target critic network, and their parameters are slowly tracked from the online network parameters through a soft update mechanism. ; ;in, (usually taken) () represents the soft update coefficient.

[0037] The global objective function is designed as a weighted negative sum of the average end-to-end latency of the task and the maximum link utilization of the network, expressed as: ;in, Indicates global reward. This represents the average end-to-end latency for all tasks. This represents the maximum link utilization of the network. and This represents the weighting coefficient, used to balance the two objectives of latency optimization and network health maintenance.

[0038] End-to-end latency of the task Due to transmission delay and computational delay It consists of two parts: Transmission delay Depending on the amount of task data Selected computing nodes and traffic allocation strategy When task traffic is distributed across multiple paths, the transmission latency is determined by the latency of the slowest path.

[0039] Calculate latency Based on task characteristics and target computing power nodes The performance is determined by the GNN model, which is used for prediction.

[0040] Maximum network link utilization Defined as the maximum utilization rate of all links in the entire network: By As a penalty term in the reward function, the agent is not only guided to minimize task latency during the learning process, but also incentivized to avoid network congestion and prevent the continuous use of the same path from causing network hotspots. This design allows the scheduling strategy to optimize task performance while also taking into account the long-term health of the network and the fairness of resource allocation.

[0041] S120. The trained lightweight reinforcement learning agents are periodically distributed to the scheduling router. The scheduling router includes a control plane and a data plane. The control plane receives local information and makes scheduling decisions, while the data plane executes the scheduling decisions. In this embodiment, the network scheduling controller periodically runs the model training process, updates the simulation environment using the latest collected historical task requirement data and network state information, and retrains the actor network of each agent. After training is completed, the controller distributes the updated actor network model parameters to each scheduling router through the control channel.

[0042] After receiving a new model, the scheduling router replaces the actor network model in the control plane with the new version at appropriate times (e.g., during off-peak hours). This periodic model update mechanism effectively copes with slow changes in the network environment and business patterns, prevents model degradation, and ensures that the scheduling strategy continues to maintain high performance.

[0043] S130, based on several access nodes of the computing power network, deploys a scheduling router.

[0044] In this embodiment, the system adopts a distributed scheduling architecture, with each access node... Deploy a scheduling router. Each scheduling router consists of two logical layers: a control plane and a data plane. The control plane incorporates a lightweight reinforcement learning agent, which is a trained neural network model capable of making rapid scheduling decisions based on local information. The data plane is responsible for the actual traffic forwarding, maintaining the forwarding table, and executing scheduling commands issued by the control plane. When a computational task arrives at an access node, the scheduling router first extracts the task's characteristic information, including the amount of task data. Information such as task type identifiers will serve as one of the inputs for the agent's decision-making. The amount of data in a task determines the time required for its transmission in the network, while the task type identifier is used to subsequently estimate the computation latency of the task on different computing nodes.

[0045] In some specific embodiments, it should also be noted that the global critic network exists only during the offline training phase and is used to guide the collaborative learning of each actor network. During the online execution phase, each agent relies solely on its local actor network for decision-making, fully preserving the low latency and high scalability advantages of the distributed framework. Agents do not need to communicate with each other or interact with the controller in real time, thus fundamentally eliminating the time overhead of information collection and decision deployment in a centralized scheduling framework.

[0046] This embodiment systematically addresses the core challenges faced by existing computing power network scheduling methods by employing a distributed scheduling architecture, a multi-agent deep reinforcement learning algorithm, a dynamically route-gated actor network, and a jointly optimized reward function design. The distributed execution paradigm achieves sub-second or even millisecond-level scheduling responses; the structured processing of the hybrid action space enables collaborative optimization of computing power node selection and traffic allocation; the introduction of a network MLU as a penalty term maintains the overall health of the network; and the introduction of a global critic network overcomes the instability of distributed learning, enabling agents to collaboratively achieve near-centralized scheduling performance even in the absence of global information.

[0047] S200: Based on the computing power network and combined with the dynamic routing gating architecture, a lightweight reinforcement learning agent in the scheduling router makes independent scheduling decisions on local observation information to obtain a composite scheduling decision. It should be noted that in some embodiments, step S200 may include S210 to S220.

[0048] S210. Each scheduling router's agent independently makes scheduling decisions based on local observation information, which includes task requirement information, local link status information, and estimated computational latency of candidate computing nodes.

[0049] In some specific embodiments, for intelligent agents Its state space It consists of the following three parts: 1) Local task information: including the amount of data currently arriving at the task. and task feature vector Task feature vectors can contain information such as task type and resource requirements. These discrete or continuous features are typically mapped into fixed-dimensional vector representations using an embedding layer. 2) Local network status: This includes the real-time status of each outgoing link of the access node. Specifically, for access nodes... Each outgoing link The agent needs to obtain the current utilization rate of the link. Link utilization is defined as the ratio of the current traffic load on a link to its capacity. ,in Indicates link The current traffic load on the network. This information can be obtained in real time through the local monitoring module of the access node without interacting with the remote controller, thus avoiding the information collection delay in a centralized architecture; 3) Estimated computation latency: for the candidate computing power node set Each node in The agent needs to obtain the estimated computational latency of the current task on this node. This estimated value is calculated using a pre-trained Graph Neural Network (GNN) model. The GNN model takes the task feature vector, the computing power parameters of the computing nodes, and the current load status of the computing nodes as input, and outputs the expected computation latency of the task on each computing node. The GNN model can capture the complex mapping relationship between task features and node performance, providing accurate latency predictions.

[0050] By combining the above three pieces of information, an intelligent agent is formed. Complete state representation ,in Represents nodes The set of directly connected neighbor nodes.

[0051] S220, the intelligent agent outputs a composite scheduling decision, including the selection of the target computing power node and the multi-path traffic allocation ratio to reach that node.

[0052] In some specific embodiments, the intelligent agent Action space It is a hybrid action space, consisting of discrete and continuous actions, expressed as: .

[0053] Discrete Actions This represents the target computing node selected for the current task. This decision requires comprehensive consideration of the computing node's computing power, current load status, and the quality of the network path leading to that node.

[0054] Continuous Actions Represents the flow allocation vector, where Indicates from the access node To the target computing node A predefined set of paths Indicates the path to be assigned The proportion of traffic on the network. The traffic allocation vector must satisfy the normalization constraint, i.e. And each component is non-negative. .

[0055] To address this hybrid action space problem, the agent's actor network employs an innovative dynamic routing gating architecture, as shown in Figure 5. This architecture comprises three core components: 1) A shared backbone network: This is a multilayer perceptron (MLP) that utilizes the agent's local observations... As input, high-dimensional state feature vectors are extracted through multiple fully connected layers and non-linear activation functions (such as ReLU). eigenvectors The dimension is typically set to 128 or 256, which is sufficient to encode key patterns in the state information. The computational expression for the backbone network is: ;in, Represents the weight matrix. This represents the bias vector. 1) Activation function; 2) Router module: This module is responsible for implementing discrete computing power node selection. It selects each candidate computing power node... Maintain a learnable centroid vector The dimension of this vector is the same as that of the feature vector. The same. Routers calculate feature vectors. With each centroid vector The dot product between these nodes is used to evaluate the suitability of selecting each computing power node in the current state; this is called the affinity score. The specific calculation process is as follows: ;in, The function maps the dot product result to The interval makes the affinity score probabilistic.

[0056] To enhance exploration capabilities and achieve load balancing, the router module also introduces a dynamic bias term based on usage frequency. This bias term is dynamically adjusted during training. For nodes with low activation frequency, the bias is increased to encourage exploration; for nodes with high activation frequency, the bias is decreased to avoid over-concentration. The final node selection is determined by the following formula: During the training phase, to ensure the differentiability of this discrete selection process and enable gradient backpropagation to update router parameters, this embodiment employs the Gumbel-Softmax technique. The operation is a differentiable approximation. Specifically, noise is first sampled from the Gumbel(0,1) distribution. Then calculate the soft selection probability: ;in, Temperature is a parameter that controls the degree of softening. When When Gumbel-Softmax approaches hard selection; when When the value is larger, a smoother selection is preferable, which is beneficial for exploration. In the early stages of training, a larger value is typically used. Value (e.g.) As training progresses, the temperature gradually anneals to a smaller value (e.g., ...). During the execution phase, use directly. The operation performs a hard selection, without requiring the Gumbel-Softmax technique.

[0057] 3) Expert Network: The system includes... Each expert network With a specific candidate computing node Binding. When the router selects a computing node. Subsequently, the corresponding expert network Activated. The expert network uses feature vectors extracted from the shared backbone network. As input, it passes through one or more fully connected layers and outputs a dimension of... vector : ;in, and It is an expert network The vector contains proprietary parameters. This vector is then normalized using the Softmax function to obtain the final traffic allocation ratio vector. The Softmax function automatically guarantees the normalization constraint and non-negativity constraint of the flow allocation ratio, that is... and .

[0058] The core advantage of this dynamic routing gating architecture lies in its structural decoupling of discrete and continuous decision-making. Routers handle discrete node selection, while expert networks are responsible for the continuous traffic allocation strategy that matches that selection. More importantly, this architecture resolves the ambiguity problem in credit allocation. When the system receives a reward signal, the gradient flows only to two places: first, the single activated expert network to optimize its traffic allocation strategy; and second, the centroid vector and related parameters corresponding to that expert in the router to optimize its node selection strategy. This mechanism precisely attributes the merits and demerits of the reward to the corresponding decision-making modules, ensuring that each link receives reasonable and independent feedback, thus effectively avoiding the local optimum dilemma.

[0059] S300: The composite scheduling decision is sent to the data plane of the scheduling router for multi-path traffic allocation, thereby realizing distributed scheduling of the computing power network. It should be noted that in some embodiments, step S300 may include: S310: The composite scheduling decision is sent to the data plane of the scheduling router; S320: Several paths from each access node to the computing power node are pre-calculated using the k-shortest path algorithm; S330: According to the multi-path traffic allocation ratio of the target computing power node in the composite scheduling decision, the traffic is allocated to several paths to realize distributed scheduling of the computing power network.

[0060] In some specific embodiments, after the agent completes the decision, it outputs a composite action. The decision is sent to the data plane of the scheduling router. Based on this decision, the data plane updates the forwarding table, configuring the traffic distribution ratio across multiple predefined paths.

[0061] In practice, the system employs source-route tunneling technology. For each source-destination pair from the access node to the computing power node... ,pass Shortest path algorithm (usually taking) Multiple paths are pre-calculated to form a path set. Each path It consists of a series of nodes and links, represented as ,in , The system configures a tunnel ID for each path and establishes forwarding rules at each node along the path to ensure that data packets with the tunnel ID are forwarded strictly according to the predefined path.

[0062] At the access node, the data plane maintains a traffic distribution table, which records how traffic arriving at each target computing power node should be allocated across multiple paths. When task traffic arrives, the access node distributes traffic according to the traffic allocation ratio output by the agent. The task data is distributed proportionally to the corresponding tunnels. For example, if Then 50% of the traffic will be forwarded through path 1, 30% through path 2, and 20% through path 3.

[0063] Intermediate forwarding nodes forward data packets according to pre-configured tunnel forwarding rules, without requiring additional scheduling decisions. The advantage of this design is that dynamic decision-making and configuration updates are only required at the access nodes, while intermediate nodes maintain static forwarding behavior, thus avoiding the huge control overhead caused by distributing routing configurations to all nodes in a centralized architecture.

[0064] Finally, in order to comprehensively and objectively verify the effectiveness, superiority and robustness of the large-scale computing power network distributed scheduling method proposed in this invention, this embodiment conducted a comprehensive experimental evaluation in a series of simulation environments.

[0065] This embodiment builds an experimental environment based on the ns-3 network simulator. ns-3 is used to construct the underlying physical network topology and simulate real data packet transmission behavior and network protocol stacks, accurately reflecting real-world characteristics such as network latency, bandwidth limitations, and queue congestion. To achieve online closed-loop interaction between the reinforcement learning algorithm and the network simulation environment, the ns3-ai component is used to implement real-time interaction between the reinforcement learning algorithm and the ns-3 environment. ns3-ai achieves efficient data exchange between the Python-implemented reinforcement learning algorithm and the C++-implemented ns-3 simulator through a shared memory mechanism.

[0066] In terms of topology selection, this embodiment selects three representative real wide area network topologies of different sizes from the SNDlib public database: Abilene: backbone network, containing 12 nodes and 15 links, representing a small-scale topology; GEANT: pan-European academic research network, containing 23 nodes and 56 links, representing a medium-scale topology; Germany50: large-scale German national network, containing 50 nodes and 88 links, representing a large-scale topology.

[0067] In each topology, three nodes are selected and deployed as computing power nodes, each allocated different computing capabilities. Four edge nodes are selected as access points; these access points are where scheduling routers are deployed, responsible for selecting computing power nodes and determining path distribution for arriving computing tasks. Client nodes are mounted on each access point to generate computing task requests. The task arrival process is modeled using a Poisson distribution, with the average arrival rate adjusted as needed to simulate different network load levels. The data volume for each task is randomly sampled from a predefined distribution.

[0068] At the forwarding layer, the k-shortest path algorithm is used to pre-calculate and configure forwarding paths for each access point-computing node pair. In the experiment, k=3, meaning that three shortest paths are pre-calculated for each source-destination pair. The system configures a source routing tunnel for each path to ensure that data packets are forwarded strictly according to the predefined paths.

[0069] To verify the effectiveness of this invention, this embodiment designs four types of two-stage discrete baseline methods for comparison. These baseline methods share the common feature of dividing scheduling decisions into two independent stages: the first stage selects computing nodes based on a single criterion, and the second stage performs path splitting under a predetermined destination node. All baseline methods use the same set of forwarding paths and the same task load as this invention to ensure fairness in the comparison.

[0070] Baseline 1 (CompMin + ECMP): In the first phase, the computing node with the lowest estimated computational latency is selected to prioritize reducing latency during the computation phase; in the second phase, Equal Cost Multipath (ECMP) is used to evenly distribute task traffic across multiple forwarding paths to the target node, i.e., the allocation ratio of each path is [missing information]. Baseline 2 (Near + ECMP): In the first phase, the computing node with the shortest hop count closest to the access point is selected to prioritize reducing transmission distance; in the second phase, ECMP is used for uniform traffic distribution. Baseline 3 (CompMin + LP): In the first phase, the computing node with the minimum estimated computational latency is selected; in the second phase, with a fixed destination node, the multi-path traffic distribution ratio is optimized using linear programming to minimize the maximum link utilization (MLU), thereby alleviating hotspots and congestion. The linear programming problem is solved using the Gurobi optimizer. Baseline 4 (Near + LP): In the first phase, the nearest computing node is selected; in the second phase, with a fixed destination node, the traffic distribution ratio is optimized using linear programming to minimize the MLU.

[0071] This embodiment uses the following three key performance indicators for evaluation: 1) Maximum Link Utilization (MLU): defined as the maximum value of the utilization of all links in the network, expressed as: ,in Indicates link Current traffic load on The following metrics are used to measure network congestion and hotspot levels: 1) Link capacity. MLU measures network congestion and hotspot levels; a lower MLU value indicates a more balanced network load and lower congestion risk. 2) Average transmission latency: Calculated by statistically analyzing the sending and receiving timestamps of task data packets. The transmission latency of a single task is defined as the average of the transmission latency of all its data packets, including propagation latency, processing latency, and queuing latency. This metric reflects the combined impact of path planning quality and queuing congestion. 3) Average end-to-end latency: Defined as the sum of transmission latency and computation latency. This is a core metric for measuring Quality of User Experience (QoE), comprehensively reflecting the combined effect of computing node selection and path planning. A lower MLU value indicates a faster perceived service response speed.

[0072] To verify the superiority of the proposed method over existing technologies, multiple comparative experiments were designed to evaluate the above indicators: 1) Evaluation of the performance of each scheme in load balancing. As shown in Figure 7, the method provided in this embodiment significantly reduces the maximum link utilization in all three topologies, especially in larger-scale scenarios with higher path diversity such as GEANT and Germany50. Baseline 1 / 2 uses static ECMP traffic splitting, ignoring the real-time load of the links, which may lead to network congestion. Although baseline 3 / 4 introduces linear programming to optimize traffic distribution, its first-stage selection of computing power nodes is rigid, based only on computing power or distance, and is unaware of network status. Once a large number of tasks are directed to the same "optimal" computing power node, even if linear programming tries its best to optimize the path, it cannot eliminate the link congestion caused by the convergence of traffic at the target node. In contrast, this embodiment introduces the maximum link utilization as a key penalty term into the reward function, not only performing fine-grained dynamic traffic splitting in data transmission, but also avoiding potential congestion areas by intelligently selecting computing power nodes. This collaboration enables the method provided in this embodiment to proactively guide traffic to areas with lower network load, thereby maintaining maximum link utilization at a healthy level and effectively improving network throughput potential and robustness; 2) Evaluate the performance of average transmission latency. As shown in Figure 8, the method provided in this embodiment maintains low transmission latency in all three simulation scenarios and is generally better than the four baselines. It is worth noting that transmission latency mainly consists of propagation latency and queuing latency. Therefore, even if baselines 2 / 4 select closer computing nodes, if network load is ignored and link congestion is triggered, queuing latency will still cause the overall transmission latency to rise sharply. In Germany50, baseline 4 has significantly better transmission latency than baseline 2 due to the use of linear programming for traffic splitting; however, its performance ceiling is still limited by the rigid computing node selection in the first stage. As shown in GEANT, when the computing node selection does not match the overall network load, simple path optimization is difficult to eliminate congestion bottlenecks. This reveals a key insight: relying solely on bottom-level path optimization cannot compensate for the defects in the upper-level computing node selection decision. This embodiment utilizes a dynamic routing gating mechanism to sensitively perceive the nonlinear impact of network congestion on transmission latency. The method intelligently balances "relatively distant, low-load nodes" with "congested nearby nodes," thereby optimizing path selection while minimizing queuing latency and achieving optimal overall transmission efficiency. 3) It evaluates the core metric affecting user experience: average end-to-end latency. As shown in Figure 9, experimental results strongly demonstrate the effectiveness of the joint optimization framework in this embodiment. End-to-end latency is composed of both transmission latency and computation latency.In small-scale topologies like Abilene, transmission distance differences are relatively limited, making computational differences more likely to dominate overall latency. Baseline 1 / 3 achieves better end-to-end performance by selecting nodes with lower computational latency, while Baseline 2 / 4, although reducing transmission latency, may send tasks to nodes with weaker computing power or higher loads, thus putting it at a disadvantage in end-to-end metrics. In GEANT and Germany50, the impact of transmission latency is more significant, making end-to-end latency more sensitive to network conditions. The method provided in this embodiment exhibits significant advantages in all scenarios. The method does not require pre-assuming whether the system bottleneck is computation or the network; instead, it automatically searches for the optimal solution in a vast mixed action space through multi-agent reinforcement learning. The method avoids blindly pursuing high computing power and introducing excessive transmission costs like Baseline 1 / 3, and also avoids sacrificing computational efficiency for proximity like Baseline 2 / 4. By simultaneously optimizing the selection of computing nodes (reducing computation latency) and multi-path traffic allocation (reducing transmission latency and avoiding congestion), this embodiment achieves deep collaboration between computing and network resources, providing optimal performance assurance for latency-sensitive AI services.

[0073] In summary, as shown in Figure 3, scheduling routers are deployed at multiple access nodes of the computing network. Each scheduling router has a built-in lightweight reinforcement learning agent that receives computing tasks arriving locally. The agents of each scheduling router independently make scheduling decisions based on local observation information, output composite scheduling decisions, and distribute the scheduling decisions to the data plane. By updating the forwarding table, the task traffic is distributed in a weighted manner across multiple predefined paths. A multi-agent deep deterministic policy gradient algorithm is used for offline training on the computing network scheduling controller. A global critic network is used to evaluate the joint actions of all agents, guiding each agent to collaboratively optimize the global objective. The trained and updated agent model is periodically distributed to each scheduling router to achieve continuous model optimization.

[0074] Therefore, the embodiments of the present invention have the following advantages over the prior art: 1) By introducing the maximum link utilization rate of the network as a penalty term in the reward function, the overall health of the network is actively maintained while optimizing task performance, effectively avoiding the formation of network hotspots. Experimental results show that on three representative real network topologies, Abilene, GEANT, and Germany50, the present invention significantly reduces the maximum link utilization rate, especially in large-scale topologies such as GEANT and Germany50. Compared with the baseline method using ECMP traffic splitting, the present invention can distribute the network load more evenly across multiple paths; compared with the baseline method using linear programming to optimize path splitting, the present invention avoids potential congestion areas from the source by intelligently selecting computing power nodes, realizing the synergistic optimization of computing power node selection and network path planning, significantly improving the robustness and long-term stability of the network; 2) The transmission latency is effectively reduced through a dynamic routing gating mechanism. Experimental results show that the present invention maintains a low transmission latency in all three simulation scenarios, and is superior to the four baseline methods overall. Compared to the ECMP uniform traffic distribution method, this invention avoids the problem of a sharp increase in queuing delay caused by ignoring network load through fine-grained dynamic traffic allocation. Compared to the method of optimizing traffic allocation with linear programming but fixing the selection of computing power nodes, this invention can keenly perceive the nonlinear impact of network congestion on transmission delay and make intelligent trade-offs between "slightly distant low-load nodes" and "congested near-end nodes", thus optimizing the overall transmission efficiency. Experimental data shows that relying solely on low-level path optimization cannot compensate for the deficiencies in upper-level computing power node selection decisions, and the joint optimization strategy of this invention has significant advantages. 3) It achieves deep joint optimization of computing resources and network resources, achieving significant performance improvement in the core indicator of user experience, average end-to-end delay. Experimental results strongly demonstrate the effectiveness of the joint optimization framework of this invention: in the Abilene small-scale topology, this invention outperforms all baseline methods; in the GEANT and Germany50 large-scale topologies, the advantages of this invention are even more obvious due to the more significant impact of transmission delay. This invention does not require prior assumptions about whether the system bottleneck is computation or network. Instead, it automatically searches for the optimal solution in a hybrid action space through multi-agent reinforcement learning. It avoids blindly pursuing high computing power and introducing excessive transmission costs, and it does not sacrifice computing efficiency for proximity access, thus providing the best performance guarantee for latency-sensitive AI services.

[0075] Referring to Figure 2, this application embodiment also provides a distributed scheduling system for a large-scale computing network, which can implement the above-mentioned distributed scheduling method for a large-scale computing network. The system includes: a first module 201, used to train lightweight reinforcement learning agents offline using a multi-agent deep deterministic policy gradient algorithm and then distribute the training to a scheduling router, and deploy the scheduling router to several access points of the computing network; a second module 202, used to make independent scheduling decisions on local observation information through lightweight reinforcement learning agents in the scheduling router based on the computing network and combined with a dynamic routing gating architecture, to obtain a composite scheduling decision; and a third module 203, used to distribute the composite scheduling decision to the data plane of the scheduling router for multi-path traffic allocation, thereby realizing the distributed scheduling of the computing network.

[0076] Furthermore, as shown in Figure 4, the system hardware module of this embodiment includes a network scheduling controller, multiple scheduling routers, and physical network forwarding devices.

[0077] The computing network scheduling controller serves as the system's offline training center, comprising the following functional modules: 1) Data acquisition module: This module periodically collects historical task demand data from each scheduling router, including task arrival time, task characteristics, and task data volume. Simultaneously, it collects network link status information through in-band network telemetry (INT) technology, including the utilization rate, available bandwidth, and latency of each link. INT technology inserts telemetry instructions into the packet header, enabling the packet to collect network status information along the way during forwarding, ultimately converging at the destination node or a dedicated telemetry collector, thus achieving fine-grained, high-frequency network status monitoring; 2) Simulation environment module: This module constructs a high-fidelity computing network simulation environment based on the collected historical data. The simulator needs to simulate the following key processes: tasks arrive at each access node according to historical statistical patterns; the agent makes scheduling decisions based on the current strategy; task traffic is transmitted in the network according to the decisions, updating the utilization rate of each link; tasks are computed after arriving at the target computing node, generating computational latency; the system calculates reward signals based on the end-to-end latency of the task and the network MLU. The simulator employs a discrete event simulation method, which can efficiently simulate the system operation process over long periods, providing sufficient empirical data for training algorithms; 3) Reinforcement learning training module: This module implements the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm, including... Each actor network and a global critic network are used. The training module collects empirical data and stores it in the replay buffer through interaction with the simulation environment module, and then iteratively updates the parameters of each network through the aforementioned training process. The training module is also responsible for auxiliary functions such as hyperparameter management, training process monitoring, and model checkpoint saving; 4) Model distribution module: This module is responsible for distributing the trained actor network model to each scheduling router.

[0078] The scheduling router, deployed at the access nodes of the computing network, is the system's distributed decision-making entity. Each scheduling router includes a control plane and a data plane: the control plane incorporates a reinforcement learning agent, which is essentially a trained actor network model. The control plane is responsible for the following functions: collecting local observation information, including task characteristics, local link status, and estimated computing latency of computing nodes; invoking the actor network model for inference and outputting scheduling decisions; issuing decision instructions to the data plane; periodically receiving model updates from the computing network scheduling controller; and reporting local task requirements and network status data for the controller to perform offline training.

[0079] The data plane is responsible for the actual traffic forwarding, including the following functions: maintaining the forwarding table and updating the forwarding table according to the scheduling instructions of the control plane; distributing the arriving task traffic through multiple paths; and performing packet forwarding based on tunnel identifiers.

[0080] Physical network forwarding devices include physical devices such as switches and routers in the network. These devices perform traffic forwarding according to pre-configured source routing tunnels. Intermediate forwarding nodes do not need to make scheduling decisions; they only look up the forwarding table based on the tunnel identifier of the data packet and forward it to the next hop, thus ensuring efficient data plane processing performance.

[0081] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0082] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A distributed scheduling method for large-scale computing networks, characterized in that, The method includes the following steps: Lightweight reinforcement learning agents are trained offline using a multi-agent deep deterministic policy gradient algorithm and then distributed to a scheduling router; the scheduling router is deployed to several access points of the computing network; based on the computing network and combined with a dynamic routing gating architecture, the lightweight reinforcement learning agents in the scheduling router independently make scheduling decisions based on local observation information to obtain composite scheduling decisions; the composite scheduling decisions are then distributed to the data plane of the scheduling router for multi-path traffic allocation, thereby achieving distributed scheduling of the computing network.

2. The method according to claim 1, characterized in that, The step of training lightweight reinforcement learning agents offline using a multi-agent deep deterministic policy gradient algorithm and then distributing the trained agents to a scheduling router, and deploying the scheduling router to several access points of the computing power network, includes: training the lightweight reinforcement learning agents offline using the multi-agent deep deterministic policy gradient algorithm; evaluating the joint actions of all lightweight reinforcement learning agents using a global critic network to obtain trained lightweight reinforcement learning agents; periodically distributing the trained lightweight reinforcement learning agents to the scheduling router, wherein the scheduling router includes a control plane and a data plane, the control plane is used to receive local information and make scheduling decisions, and the data plane is used to execute the scheduling decisions; and deploying the scheduling router based on several access nodes of the computing power network.

3. The method according to claim 2, characterized in that, The step of offline training of the lightweight reinforcement learning agent using the multi-agent deep deterministic policy gradient algorithm and evaluating the joint actions of all lightweight reinforcement learning agents using a global critic network to obtain the trained lightweight reinforcement learning agent includes: offline training of the lightweight reinforcement learning agent using the multi-agent deep deterministic policy gradient algorithm, which includes several actor networks and a global critic network, each actor network corresponding to one lightweight reinforcement learning agent; acquiring the local observation information of the lightweight reinforcement learning agent and inputting it into the actor network to obtain the actions of the lightweight reinforcement learning agent; acquiring the hidden state information of the lightweight reinforcement learning agent using an in-band network telemetry method and combining it with the... The actions and local states of a lightweight reinforcement learning agent are used to construct global information, which is then input into the global critic network to obtain the expected cumulative reward for the actions of the lightweight reinforcement learning agent. An experience replay pool is constructed by combining the actions and local states of the lightweight reinforcement learning agent, the expected cumulative reward, and the actions of the lightweight reinforcement learning agent at the next time step. Experience sample data is randomly extracted from the experience replay pool by the scheduling controller in the computing power network, and the global critic network is updated by minimizing the temporal difference error. The actor network is updated based on the expected cumulative reward, a global objective function is designed, and the lightweight reinforcement learning agent is trained based on the updated global critic network and the updated actor network to obtain the trained lightweight reinforcement learning agent.

4. The method according to claim 3, characterized in that, The global objective function is a weighted negative sum of the average end-to-end latency of the task and the maximum link utilization of the network. The average end-to-end latency of the task includes transmission latency and computation latency. The maximum link utilization of the network represents the maximum value of the utilization of all links in the computing network. The specific expression of the global objective function is as follows: In the above formula, Indicates global reward. This represents the average end-to-end latency for all tasks. This represents the maximum link utilization of the network. and This represents the weighting coefficient, used to balance the two objectives of latency optimization and network health maintenance.

5. The method according to claim 1, characterized in that, The local observation information includes task requirement information, local link status information, and estimated computational latency of candidate computing nodes. Specifically: the task requirement information includes the amount of data currently arriving at the task and the task feature vector; the local link status information represents the real-time status of each outgoing link of the access node; and the estimated computational latency of candidate computing nodes indicates that for each node in the candidate computing node set, the lightweight reinforcement learning agent needs to obtain the estimated computational latency of the current task on that node.

6. The method according to claim 1, characterized in that, The composite scheduling decision includes a discrete action space and a continuous action space, wherein: the discrete action space represents the target computing power node selected by the current task; and the continuous action space represents the multi-path traffic allocation ratio to the target computing power node.

7. The method according to claim 1, characterized in that, The dynamic routing gating architecture includes a shared backbone network, router modules, and an expert network. The shared backbone network represents a multilayer perceptron, which takes local observation information from the lightweight reinforcement learning agent as input and extracts a high-dimensional state feature vector through several fully connected layers and nonlinear activation functions. The router module introduces a dynamic bias term based on usage frequency and maintains corresponding centroid vectors for candidate computing power nodes. It then performs a dot product calculation with the high-dimensional state feature vector and selects the candidate computing power nodes based on the calculation result using the Gumbel-Softmax function. The expert network is bound to each candidate computing power node, takes the high-dimensional state feature vector as input, outputs a dimension vector through fully connected layers, and obtains the multipath traffic allocation ratio for the target computing power node by normalizing the dimension vector.

8. The method according to claim 7, characterized in that, The specific expression for the Gumbel-Softmax function is as follows: In the above formula, 、 They represent experts respectively. 、 Affinity score, 、 Representing nodes respectively 、 Noise sampled from the Gumbel(0,1) distribution. Indicates temperature parameter, Represents the set of candidate computing power nodes. This indicates traversing the set of candidate computing power nodes. All nodes in This represents the soft selection probability after Gumbel-Softmax processing.

9. The method according to claim 1, characterized in that, The step of sending the composite scheduling decision to the data plane of the scheduling router for multi-path traffic allocation to achieve distributed scheduling of the computing power network includes: sending the composite scheduling decision to the data plane of the scheduling router; pre-calculating several paths from each access node to the computing power node using the k-shortest path algorithm; and allocating traffic to several of the paths according to the multi-path traffic allocation ratio of the target computing power node in the composite scheduling decision to achieve distributed scheduling of the computing power network.

10. A distributed scheduling system for a large-scale computing network, characterized in that, The system comprises: a first module, used to train lightweight reinforcement learning agents offline using a multi-agent deep deterministic policy gradient algorithm and then distribute the training to a scheduling router, and deploy the scheduling router to several access points of the computing power network; a second module, used to, based on the computing power network and combined with a dynamic routing gating architecture, use the lightweight reinforcement learning agents in the scheduling router to make independent scheduling decisions on local observation information to obtain composite scheduling decisions; and a third module, used to distribute the composite scheduling decisions to the data plane of the scheduling router for multi-path traffic allocation, thereby realizing distributed scheduling of the computing power network.

Images