Scheduling method, device and computer equipment for transmission tasks of computing power network

Abstract

The present invention discloses a method, apparatus, and computer equipment for scheduling transmission tasks of a computing power network. The method comprises: obtaining the network status of the computing power network based on a software-defined network (SDN), wherein the network status includes the status of transmission nodes and transmission links in the computing power network, and the computing power network includes faulty transmission nodes and / or faulty transmission links; obtaining the target transmission task of the computing power network; inputting the network status and the target transmission task into a target network model to obtain a scheduling route for the target transmission task. The present invention solves the technical problem in related technologies of introducing a fault-tolerant function into the TSN scheduling mechanism relied upon by the computing power network, and of having low scheduling efficiency when calculating the transmission path from a large number of redundant paths.

Description

Technical Field

[0001] The present invention relates to the field of electric power technology, and in particular to a method, device and computer equipment for scheduling transmission tasks of a computing power network. Background Art

[0002] A computing network is a new type of information infrastructure that allocates and flexibly schedules computing, storage, and network resources across the cloud, network, and edge based on business needs. As a new paradigm for cloud-network convergence, computing networks have garnered widespread attention across the industry. In particular, with the advancement of cutting-edge network technologies such as cloud native, time-sensitive networking, and artificial intelligence, computing networks are also emerging with new possibilities, such as cloud native computing networks, deterministic computing networks, and self-intelligent computing networks. These initiatives aim to enhance computing network orchestration and scheduling flexibility, enhance the time-sensitivity of computing and transmission, and enhance intelligent decision-making and governance. Deterministic computing networks encompass deterministic transmission capabilities, which must meet the specific transmission and computational latency requirements of latency-sensitive services on computing networks. This relies on the low-latency, highly jitter-resistant communication services provided by time-sensitive networks (TSN), based on fundamental capabilities such as network-wide clock synchronization, time-aware shaping, and round-robin forwarding. However, if a TSN network experiences device or link failures, it can result in prolonged loss of time-sensitive data traffic on the computing network, severely impacting the normal operation of computing network services. Therefore, in order to ensure the computing network determinism and the stability and reliability of TSN, the TSN scheduling mechanism that the computing network relies on needs to introduce fault tolerance functions.

[0003] When considering fault tolerance, it is often achieved by transmitting multiple copies of data across multiple spatially redundant paths. However, this type of fault tolerance requires a significant amount of time to precompute candidate redundant paths to build the redundant path set, resulting in low network resource utilization and impacting the upper limit of normal scheduling performance.

[0004] Therefore, in the related art, a fault-tolerant function is introduced into the TSN scheduling mechanism that the computing power network relies on. When calculating the transmission path from a large number of redundant paths, there is a problem of low scheduling efficiency.

[0005] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0006] Embodiments of the present invention provide a method, apparatus, and computer device for scheduling transmission tasks of a computing power network, so as to at least solve the technical problem in related technologies of introducing fault tolerance into the TSN scheduling mechanism relied upon by the computing power network, and of low scheduling efficiency when calculating the transmission path from a large number of redundant paths.

[0007] According to one aspect of an embodiment of the present invention, a scheduling method for transmission tasks of a computing power network is provided, comprising: obtaining a network status of the computing power network based on a software-defined network (SDN), wherein the network status includes the status of transmission nodes and the status of transmission links in the computing power network, and the computing power network includes faulty transmission nodes and / or faulty transmission links; obtaining a target transmission task of the computing power network; inputting the network status and the target transmission task into a target network model to obtain a scheduling route for the target transmission task, wherein the target network model reconstructs the node status characteristics of the transmission nodes in the computing power network and the link flow characteristics of the transmission links based on a multi-headed graph attention residual network, and adopts an attention mechanism to adjust the weights of the node status characteristics and the link flow characteristics; based on the adjusted weights of the node status characteristics and the link flow characteristics, adopting a duel network to select a transmission link for the target transmission task, wherein the scheduling route is composed of multiple transmission links obtained hop by hop based on the multi-headed graph attention residual network and the duel network.

[0008] Optionally, the network status of the computing power network obtained based on the software-defined network SDN includes: obtaining the topology of the computing power network; obtaining a set of transmission nodes within the topology, and a set of transmission links within the topology; determining the status of each transmission node in the transmission node set, including at least one of the following: the capacity upper limit of the port of the transmission node, the different priority sending queues virtualized by the circular queue and forwarding queue on the port of the transmission node through the SDN; the resource utilization of the transmission node, the resource utilization of the sending queue, the resource utilization of the computing power network, the status of the traffic already scheduled on the transmission node, the status of the traffic being scheduled on the transmission node, and whether the transmission node is a faulty node; determining the status of each transmission link in the transmission link set, including at least one of the following: the transmission rate of the transmission link, the maximum data frame length of the transmission link, the propagation delay of the transmission link, the synchronization delay of the transmission link, the load rate of the transmission link, and whether the transmission link is a faulty link.

[0009] Optionally, obtaining the target transmission task of the computing power network includes: obtaining multiple transmission tasks in the computing power network, and the priorities of the multiple transmission tasks; based on the priorities of the multiple transmission tasks, selecting a candidate transmission task with the highest priority from the multiple transmission tasks; in the case where there are multiple candidate transmission tasks with the highest priority, selecting the candidate transmission task with the earliest task request time as the target transmission task.

[0010] Optionally, obtaining the priorities of the multiple transmission tasks includes: determining that the priority of the transmission task corresponding to the flow that needs to be rerouted is higher than the priority of the transmission task corresponding to the normal flow.

[0011] Optionally, the multi-head graph attention residual network in the target network model includes multiple layers, each layer has multiple independent attention mechanisms, and each attention mechanism is used to adaptively adjust the weight of each neighbor link information according to the traffic scheduling status indicated by the current layer input, selectively focus on key information, and perform weighted fusion of information transmission between different links to obtain output.

[0012] Optionally, before inputting the network state and the target transmission task into the target network model to obtain the scheduling route of the target transmission task, it also includes: obtaining a sample set for training the initial network model; based on the sample set, using the MGAR-based priority experience replay duel dual-depth q network PER-MD3QN algorithm to train the initial network model to obtain the target network model, wherein the PER-MD3QN algorithm includes three elements: state, action, and reward, wherein the state is the state of the transmission node and the state of the transmission link in the computing power network, and the action is to output a suitable route for each flow according to the state and send it to the corresponding sending queue priority, and the reward is generated by the reward value for successful scheduling, the reward for traffic scheduling and transmission in the intermediate link, and the penalty value for scheduling failure.

[0013] Optionally, based on the sample set, the MGAR-based prioritized experience replay duel dual-depth Q network PER-MD3QN algorithm is used to train the initial network model to obtain the target network model, including: based on the sample set, the PER-MD3QN algorithm and the Q learning training method are used to train the initial network model to obtain the target network model, wherein, when the Q learning training method is used, the Q value function is decomposed into a state value function and an advantage function, and the state value function is updated based on the time difference error, and the time difference error is the difference between the function value of the current state and the function value of the next state.

[0014] Optionally, based on the sample set, the MGAR-based Prioritized Experience Replay Duel Dual-Depth Q Network PER-MD3QN algorithm is used to train the initial network model to obtain the target network model, including: based on the sample set, the MGAR-based Prioritized Experience Replay Duel Dual-Depth Q Network PER-MD3QN algorithm is used to train the initial network model to obtain an intermediate network model; determining constraints for optimizing the intermediate network model, wherein the constraints include at least one of the following: routing hop constraints, path constraints, optional queues and capacity constraints; based on the constraints, optimizing the intermediate network model to obtain the target network model.

[0015] According to another aspect of the present invention, a scheduling device for transmission tasks of a computing power network is provided, comprising: a first acquisition module for acquiring the network status of the computing power network based on a software-defined network (SDN), wherein the network status includes the status of the transmission nodes and the status of the transmission links in the computing power network, and the computing power network includes faulty transmission nodes and / or faulty transmission links; a second acquisition module for acquiring the target transmission task of the computing power network; a processing module for inputting the network status and the target transmission task into a target network model to obtain a scheduling route for the target transmission task, wherein the target network model reconstructs the node status characteristics of the transmission nodes in the computing power network and the link flow characteristics of the transmission links based on a multi-headed graph attention residual network, and adopts an attention mechanism to adjust the weights of the node status characteristics and the link flow characteristics; based on the adjusted weights of the node status characteristics and the link flow characteristics, a duel network is adopted to select the transmission link for the target transmission task, wherein the multiple transmission links obtained hop by hop based on the multi-headed graph attention residual network and the duel network constitute the scheduling route.

[0016] According to another aspect of the present invention, a computer device is provided, comprising: a memory and a processor, wherein the memory stores a computer program; the processor is configured to execute the computer program stored in the memory, wherein when the computer program is executed, the processor executes any one of the above-described methods for scheduling transmission tasks of a computing power network.

[0017] In an embodiment of the present invention, the network status of the computing power network is obtained based on SDN, and the target network model is used to reconstruct the node status characteristics of the transmission nodes in the computing power network and the link flow characteristics of the transmission link based on the multi-head graph attention residual network, and the attention mechanism is used to adjust the weights of the node status characteristics and the link flow characteristics; based on the adjusted weights of the node status characteristics and the link flow characteristics, a duel network is used to select the transmission link for the target transmission task, and by combining SDN with the target network model, the purpose of intelligently and efficiently determining the transmission link corresponding to the transmission task based on the status of the computing power network is achieved, thereby achieving the technical effect of efficient fault tolerance in the event of a failure in the computing power network, compared with the method of determining the transmission link by using redundant paths, and thus solving the technical problem of low scheduling efficiency when calculating the transmission path from a large number of redundant paths in the related technology for introducing a fault-tolerant function into the TSN scheduling mechanism on which the computing power network relies. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0019] Figure 1 is a flowchart of a method for scheduling transmission tasks of a computing power network according to an embodiment of the present invention;

[0020] Figure 2 This is a system architecture diagram of large-scale TSN network abnormal state fault tolerance provided in an optional embodiment of the present invention;

[0021] Figure 3 is a schematic diagram of a PER-MD3QN network provided according to an embodiment of the present invention;

[0022] Figure 4 is a schematic diagram of a graph attention coefficient provided according to an embodiment of the present invention;

[0023] Figure 5 is a flow chart of the PER-MD3QN algorithm provided according to an embodiment of the present invention;

[0024] Figure 6 is a schematic diagram of the change of average reward with the number of training steps during the simulation process of an embodiment of the present invention;

[0025] Figure 7 is a schematic diagram showing changes in the scheduling success rate with the number of time-sensitive flows during the simulation process of an embodiment of the present invention;

[0026] Figure 8is a schematic diagram showing changes in the number of high-load links versus the number of time-sensitive flows during a simulation of an embodiment of the present invention;

[0027] Figure 9 2 is a schematic diagram showing changes in the scheduling success rate under different numbers of faulty nodes during the simulation process of an embodiment of the present invention;

[0028] Figure 10 is a schematic diagram of changes in the number of high-load links under different numbers of fault nodes during the simulation process of an embodiment of the present invention;

[0029] Figure 11 2. It is a schematic diagram showing the change of the scheduling success rate under different numbers of MGBs during the simulation process of an embodiment of the present invention;

[0030] Figure 12 is a schematic diagram of changes in the number of high-load links under different numbers of MGBs during the simulation process of an embodiment of the present invention;

[0031] Figure 13 This is a structural block diagram of a scheduling device for transmission tasks of a computing power network provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] To facilitate the description of this application, the terms involved in this application are explained below.

[0035] Computing power network: refers to a new type of network architecture that provides powerful computing power as a service through the integration of cloud computing and network technologies.

[0036] Time-sensitive networks (TSN): A network communication architecture designed for industrial applications that require determinism and low latency, with the ability to guarantee end-to-end latency within certain upper and lower bounds.

[0037] IEEE 802.1Qch: A network communication standard for TSN that supports time-deterministic network communication and ensures the determinism of transmission latency through circular queue forwarding.

[0038] Gating cycle: In network communications, a time unit used to unify different transmission cycles.

[0039] E2E (end to end): refers to the entire transmission process from one network terminal to another.

[0040] CQF Queues: A circular queue forwarding mechanism used for traffic shaping and scheduling in network communications. CQF features a ping-pong queue on the switch port, alternating between two queues for transmit and receive. Hardware-scheduled time slots—the alternating periods between queue flips—are typically obtained by dividing the gated cycle into fixed time slots. CQF ensures that all traffic received by the switch in one time slot is transmitted in the next. Therefore, the upper and lower bounds of E2E latency are proportional to the number of switches in the transmission path, ensuring latency determinism.

[0041] Software-defined networking (SDN): Its core concept is to dynamically control and manage network resources through software programs, thereby improving network flexibility, programmability, and intelligence. In traditional network architectures, the control and management functions of network devices (such as routers and switches) are embedded in the device's firmware or operating system, limiting the network's flexibility and scalability. SDN separates the network control layer (responsible for network management and decision-making) from the data forwarding layer (responsible for actual data transmission), making network control more flexible and centralized.

[0042] SDN controller: In software-defined networking, a device or software used to control network behavior and manage network resources.

[0043] CUC (centralized user configuration): Centralized user configuration controller, used to analyze user transmission requests and transfer them to the network configuration for processing.

[0044] CNC (centralized network configuration): Centralized network configuration, responsible for processing user transmission requests from CUC.

[0045] ILP (integer linear programming): Integer linear programming is a mathematical method used to find the integer-valued optimal solution to a linear relationship.

[0046] SMT (satisfiability modulo theories): Satisfiability modulo theories, a method for solving the satisfiability problem of logical formulas.

[0047] PER-MD3QN: A Prioritized Experience Replay algorithm based on Multi-head GATResNet and Dueling Double Deep Q-Network for solving the scheduling problem.

[0048] MGAR (multi-head graph attention residual network): Multi-head graph attention residual network, a deep learning model that uses a multi-head graph attention mechanism and a residual network structure to process graph structured data.

[0049] MGB (multi-head graph attention networks based block): a network block based on the multi-head graph attention mechanism.

[0050] D3QN: Deep Q Network, a deep reinforcement learning algorithm that combines dueling networks and dual deep Q networks for decision-making process.

[0051] DRL (deep reinforcement learning): Deep reinforcement learning, which combines deep learning and reinforcement learning methods to optimize the decision-making of intelligent agents.

[0052] GCN (graph convolutional network): Graph convolutional network, a deep learning model for graph structured data.

[0053] HDSR: A heuristic-based dynamic scheduling and routing algorithm.

[0054] HSFS: A heuristic-based shared fault-tolerant segment algorithm.

[0055] DRLS: A hop-by-hop scheduling algorithm based on deep reinforcement learning.

[0056] DIGS: A hop-by-hop scheduling algorithm based on deep reinforcement learning and integrated GCN.

[0057] Transmission task: The data stream that needs to be transmitted in the network, including attributes such as source node, destination node, priority, maximum delay, start time, required bandwidth, and transmission period.

[0058] MTU (maximum transmission unit): Maximum transmission unit, the maximum length of a single data frame that can be transmitted in the network.

[0059] TD (temporal-difference): Temporal difference learning, a method for estimating the error of the learning value function.

[0060] To facilitate the description of this application, the meanings of the letters involved in this application are explained below.

[0061] G = {V, E}: represents the overall network topology of the transport layer, which has a set of switching nodes V and a set of transmission links E.

[0062] V={v a ,a∈N}: represents a=1,2,3,...,N switching nodes v in the corresponding topology a A collection of .

[0063] E={e a,b |a,b∈N,a≠b}: represents the physical link e between nodes a and b a,b A collection of .

[0064] ebw a,b Indicates link e a,b Current remaining bandwidth.

[0065] A=R N×N The adjacency matrix represents the network topology. a,b =1 represents e a,b ∈E, when A a,b =0 means

[0066] P represents the upper limit of each node's port number.

[0067] M=R N×P An adjacency table representing the neighbor node IDs corresponding to each port of each node in G.

[0068] m a,p Represents the neighborhood of node a corresponding to port p, where

[0069] FL represents the number of transmission tasks.

[0070] flowi represents the i-th transmission task.

[0071] sf t Indicates the number of flows that were successfully transmitted at time t.

[0072] F represents the set of all transmission tasks.

[0073] src i Indicates the source node of the transmission.

[0074] dst i Indicates the destination node ID.

[0075] pr i Indicates flow i priority.

[0076] d i Indicates flow i The maximum delay.

[0077] t i Indicates flow i The starting time slot.

[0078] bw i Indicates flow i The required bandwidth.

[0079] prd i Indicates flow i Sending cycle.

[0080] φ i Indicates flow i The planned transmission path.

[0081] T represents the gate cycle duration.

[0082] sl represents the hardware scheduling slot length.

[0083] s represents the link sending rate.

[0084] d trans Indicates the link propagation delay.

[0085] d sync Indicates the link synchronization delay.

[0086] ξ represents the upper limit of the port queue capacity.

[0087] Indicates the queue usage capacity of priority pr at port p of node a.

[0088] represents the resource utilization of the queue of port p of node a in time slot t, which can be expressed by the formula Calculate this value.

[0089] represents the resource utilization of node a in time slot t, which can be expressed by the formula Calculate this value.

[0090] Represents the overall network resource utilization, which can be expressed by the formula Calculate this value.

[0091] It represents the usage status of node a in time slot t, such as queue bandwidth and other resources, which can be expressed by the formula Calculate this value.

[0092] It represents the state of the scheduled traffic of node a in time slot t according to the destination node, priority, data frame and occupied bandwidth, which can be expressed by the formula Calculate this value.

[0093] It represents the state of the traffic that node a needs to process in time slot t, which can be expressed by the formula Calculate this value.

[0094] Indicates that node a is a faulty node or flow in time slot t i The node where the transmission path already exists.

[0095] Indicates the shortest number of hops from node a to the destination node in time slot t.

[0096] rts t Indicates the overall current state of the network at time slot t, where this value can be calculated using the formula.

[0097] Represents the h-th graph attention mechanism in the k-th layer MGB in the multi-head graph attention residual network.

[0098] represents the feature matrix of the k-th layer MGB input at time slot t.

[0099] represents the feature of the i-th node input to the k-th layer MGB at time slot t.

[0100] Represents the shared linear transformation matrix in the attention mechanism h.

[0101] ak k.h Node i and node j will pass through High-dimensional features after transformation and splicing Mapped to a matrix of real numbers, where the value can be calculated using the formula.

[0102] Represents an activation function.

[0103] Represents the normalized attention coefficient of link node i and adjacent chain nodes, which can be obtained by the formula Calculate this value.

[0104] The feature matrix representing the output of the h-th attention mechanism in the k-th layer MGB at time slot t is based on The result of weighted summation can be obtained by the formula Calculate this value.

[0105] Represents the k+1th layer MGB input, which is the output of the kth layer MGB and the kth layer MGB input obtained by residual connection The sum of which can be obtained by the formula Calculate this value.

[0106] e2e i Indicates the end-to-end delay.

[0107] H i Indicates flow i The number of hops left from the current node to the destination node can be calculated by the formula Calculate this value.

[0108] φ i Indicates flow i Determined transmission path.

[0109] Next represents the set of next hop nodes that the current node can select, which can be obtained by the formula Calculate this value.

[0110] successRate indicates the scheduling success rate.

[0111] hll t Indicates the number of high-load links.

[0112] thd represents the threshold value for calculating a high-load link.

[0113] S t Represents the input state of the PER-MD3QN algorithm.

[0114] act tRepresents the action space of the PER-MD3QN algorithm, which is a set of N*prMax long one-hot vectors.

[0115] a n,pr Indicates the scheduling action for the send queue pr of node n.

[0116] Represents the reward function of the PER-MD3QN algorithm, which can be expressed by the formula Calculate this value.

[0117] λ represents the reward coefficient of the reward function at the intermediate node scheduling.

[0118] Z represents the reward value as a positive number in the reward function.

[0119] Represents the network in state space Action space act t , network parameters θ t The Q-value function under .

[0120] Represented in state space Action space act t , network parameters θ t The state value function (Value function) under .

[0121] Represented in state space Action space act t , network parameters θ t Advantage function under .

[0122] δ represents the learning rate, which ranges from (0,1].

[0123] evalQ t (S t ,act t θ t,eval ) represents the evaluation network.

[0124] θ t,eval Indicates the network parameters corresponding to the evaluation network.

[0125] targetQ t (S t ,act t θ t,target ) indicates the target network.

[0126] θ t,target Indicates the network parameters corresponding to the target network.

[0127] y tRepresents the value function estimate of the current state by PER-MD3QN, which can be obtained by the formula Calculate this value.

[0128] ζ t The temporal-difference error is the calculation of the difference between the value function estimate of the current state and the value function estimate of the next state. The larger the error, the more learning and updating the experience is needed. t =y t -evalQ t (S t ,act t θ t,eval ) to calculate the value.

[0129] per t Indicates that priority experience playback is based on ζ t Give the corresponding experience playback priority, which can be achieved through the formula per t =|ζ t The value can be calculated by using the formula | + υ.

[0130] PER t Represents the sampling probability of the corresponding experience replay priority, which can be obtained by the formula PER t =(per t ) α / ∑(per t ) α Calculate this value.

[0131] υ represents a small constant to prevent ζ t The probability of extracting 0 experience points is 0.

[0132] α represents the priority per t The degree of use of , where 0 corresponds to the uniform situation.

[0133] Represents importance sampling, which is used to correct the priority error introduced, which can be obtained by formula w t =(per t / min(per t )) β Calculate this value.

[0134] β represents the degree of correction.

[0135] L(θ t,eval ) represents the minimization loss function, which is used to learn the optimal network parameters θ t . Which can be obtained by the formula Calculate this value.

[0136] Represents the stochastic gradient descent used to optimize the network, which can be expressed by the formula Calculate this value.

[0137] τ represents the coefficient for soft updating of the parameters of the evaluation network.

[0138] According to an embodiment of the present invention, a method embodiment of a method for scheduling transmission tasks of a computing power network is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0139] Figure 1 : is a flowchart of a method for scheduling transmission tasks of a computing power network according to an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0140] Step S102: Acquire the network status of the computing network based on the software-defined network (SDN), wherein the network status includes the status of transmission nodes and transmission links in the computing network, and the computing network includes faulty transmission nodes and / or faulty transmission links.

[0141] Step S104, obtaining the target transmission task of the computing power network;

[0142] In step S106, the network status and the target transmission task are input into the target network model to obtain the scheduling route of the target transmission task, wherein the target network model reconstructs the node status characteristics of the transmission nodes and the link flow characteristics of the transmission links in the computing network based on the multi-head graph attention residual network, and uses the attention mechanism to adjust the weights of the node status characteristics and the link flow characteristics; based on the adjusted weights of the node status characteristics and the link flow characteristics, a duel network is used to select the transmission link for the target transmission task, wherein the multiple transmission links obtained hop by hop based on the multi-head graph attention residual network and the duel network constitute the scheduling route.

[0143] Through the above steps, the network status of the computing power network is obtained based on SDN, and the node status characteristics of the transmission nodes and the link flow characteristics of the transmission links in the computing power network are reconstructed based on the multi-head graph attention residual network using the target network model, and the attention mechanism is used to adjust the weights of the node status characteristics and the link flow characteristics; based on the adjusted weights of the node status characteristics and the link flow characteristics, the duel network is used to select the transmission link for the target transmission task. By combining SDN with the target network model, the purpose of intelligently and efficiently determining the transmission link corresponding to the transmission task based on the status of the computing power network is achieved, thereby achieving the technical effect of efficient fault tolerance in the event of a failure in the computing power network, compared with the method of determining the transmission link by using redundant paths, and thus solving the technical problem of low scheduling efficiency when introducing fault tolerance into the TSN scheduling mechanism relied on by the computing power network in related technologies and calculating the transmission path from a large number of redundant paths.

[0144] As an optional embodiment, when obtaining the network status of the computing network based on the software-defined network SDN, a variety of methods can be used. For example, the topology of the computing network can be obtained first; then, a set of transmission nodes in the topology and a set of transmission links in the topology are obtained; determining the status of each transmission node in the transmission node set includes at least one of the following: the capacity upper limit of the port of the transmission node, the different priority sending queues virtualized by the circular queue and forwarding queue on the port of the transmission node through the SDN; the resource utilization of the transmission node, the resource utilization of the sending queue, the resource utilization of the computing network, the status of the traffic scheduled on the transmission node, the status of the traffic being scheduled on the transmission node, and whether the transmission node is a faulty node; determining the status of each transmission link in the transmission link set includes at least one of the following: the transmission rate of the transmission link, the maximum data frame length of the transmission link, the propagation delay of the transmission link, the synchronization delay of the transmission link, the load rate of the transmission link, and whether the transmission link is a faulty link. It should be noted that the status of the above-mentioned transmission nodes and the status of the transmission link are only examples, and other parameters used to determine the scheduling route may also be included in the scope of this application.

[0145] As an optional embodiment, when obtaining the target transmission task of the computing power network, a variety of methods can also be used. For example, multiple transmission tasks in the computing power network and the priorities of the multiple transmission tasks can be obtained first; based on the priorities of the multiple transmission tasks, the candidate transmission task with the highest priority is selected from the multiple transmission tasks; when there are multiple candidate transmission tasks with the highest priority, the candidate transmission task with the earliest task request time is selected as the target transmission task. Among them, obtaining the priorities of multiple transmission tasks can also be based on a variety of conditions. For example, for nodes or transmission links that have failed in the computing power network, since the failure point will affect the selection of other links, the transmission tasks of the flows corresponding to these failure points can be set higher. That is, when obtaining the priorities of multiple transmission tasks, it can be determined that the priority of the transmission task corresponding to the flow that needs to be rerouted is higher than the priority of the transmission task corresponding to the normal flow. By giving priority to the transmission task of the flow corresponding to the failure point, since there is no need to adjust back and forth repeatedly, it can effectively improve processing efficiency.

[0146] As an optional embodiment, the multi-head graph attention residual network in the target network model comprises multiple layers, each with independent attention mechanisms. Each attention mechanism adaptively adjusts the weights of neighboring link information based on the traffic scheduling status indicated by the current layer's input, selectively focusing on key information and performing a weighted fusion of information transferred between different links to produce an output. Through the network's attention mechanism, the key information involved in determining scheduling routes is adaptively learned, making this approach both fast and accurate.

[0147] As an optional embodiment, before inputting the network state and target transmission task into the target network model to obtain the scheduling route for the target transmission task, the method further includes: obtaining a sample set for training the initial network model; and, based on the sample set, training the initial network model using the MGAR-based Prioritized Experience Replay Duel Dual Deep Q-Network (PER-MD3QN) algorithm to obtain the target network model. The PER-MD3QN algorithm includes three elements: state, action, and reward. The state refers to the state of the transmission nodes and transmission links in the computing network. The action is to output an appropriate route for each flow based on the state and send it to the corresponding send queue priority. The reward is generated by the reward value for successful scheduling, the reward for scheduled transmission of traffic on intermediate links, and the penalty value for scheduling failure. The PER-MD3QN algorithm is used to train the initial network model. The action maximizes real-time network and flow information, outputs an appropriate route for each flow, and sends it to the corresponding send queue priority, thereby maximizing network resource utilization and more rationally forwarding flows. The reward for the intermediate links needs to ensure network resource load balancing while maximizing the scheduling success rate, limiting the occupancy of queue resources exceeding a high load threshold.

[0148] As an optional embodiment, based on the sample set, the MGAR-based Prioritized Experience Replay Duel Dual-Deep Q Network PER-MD3QN algorithm is used to train the initial network model to obtain the target network model, including: based on the sample set, the PER-MD3QN algorithm and the Q-learning training method are used to train the initial network model to obtain the target network model, wherein, when the Q-learning training method is used, the Q-value function is decomposed into a state value function and an advantage function, and the state value function is updated based on the time difference error, and the time difference error is the difference between the function value of the current state and the function value of the next state. Among them, Q-learning is a classic reinforcement learning algorithm, and its core idea is to guide the intelligent agent to make the best action in different states by learning a value function (Q function). Decomposing the Q-value function into a state value function and an advantage function can simultaneously learn the value of the state and the pros and cons of different actions.

[0149] As an optional embodiment, based on the sample set, the MGAR-based Priority Experience Replay Duel Dual-depth Q Network PER-MD3QN algorithm is used to train the initial network model to obtain the target network model, including: based on the sample set, the MGAR-based Priority Experience Replay Duel Dual-depth Q Network PER-MD3QN algorithm is used to train the initial network model to obtain an intermediate network model; determining the constraints for optimizing the intermediate network model, wherein the constraints include at least one of the following: routing hop constraints, path constraints, optional queues and capacity constraints; based on the constraints, optimizing the intermediate network model to obtain the target network model. Among them, the routing hop constraint considers the maximum number of paths corresponding to the worst case. The path constraint can limit various requirements for the path, for example, each node b in the next node set next should have a link pointed to by node a, and the remaining bandwidth capacity of the link should not be less than bw i The selected node should not be a faulty node or an invalid node that forms a link loop, and the number of hops from the node to the destination node should not exceed the currently allowed limit. By using these constraints for model optimization, the success rate is improved while ensuring load balancing of scheduling tasks and reducing the number of high-load links. This reduces the number of flows that require retransmission scheduling when node failures and link interruptions occur, and reserves network resources for retransmission scheduling.

[0150] When faced with network anomalies such as node failures or link outages, commonly used theoretical strategies often struggle to balance scheduling performance and fault tolerance. Scheduling mechanisms based on mathematical strategies such as heuristic algorithms, integer linear programming (ILP), satisfiability modulo theories (SMT), and optimization modulo theory often achieve fault tolerance by transmitting multiple data copies along multiple spatially redundant paths. However, this type of fault tolerance requires significant time to precompute candidate redundant paths to construct the redundant path set, resulting in low network resource utilization and impacting the upper limit of normal scheduling performance. Even with precomputation and storage to minimize response time, it remains static and can only address anticipated failures in small, closed networks for traffic flows where information is known and redundant paths have been precomputed for the source and destination nodes. Furthermore, fault tolerance fails when link failures occur on all redundant transmission paths with a limited number of data copies. Therefore, in open large-scale TSN scenarios, the online scheduling mechanism based on mathematical strategies only reroutes and schedules traffic related to abnormal node links in order to compromise fault tolerance and scheduling success rate. However, the transmission recovery of time-sensitive flow data still requires a long response time.

[0151] To ensure the fault tolerance of computing power network transmission in abnormal scenarios such as node failure and link interruption, and the scheduling performance in normal scenarios, this embodiment provides a TSN fault-tolerant time-sensitive flow scheduling scheme, namely, a Scheduling Mechanism for Recovering Gaping Data Time-Sensitive Traffic in Computer Power Network (SRGD), which is detailed as follows.

[0152] This embodiment provides a fault-tolerant TSN system architecture based on the IEEE 802.1Qch standard and software-defined networking (SDN). This architecture incorporates the IEEE 802.1Qch standard and software-defined networking (SDN) concepts. The SDN controller captures the link status of TSN nodes, makes online routing decisions for normal traffic, and prioritizes rerouting for abnormal traffic affected by faults.

[0153] A joint routing and scheduling optimization model was established. Aiming to achieve a higher scheduling success rate, the model used traffic latency and network resources as constraints and employed the PER-MD3QN algorithm to solve the joint routing and scheduling problem.

[0154] A PER-MD3QN (MGAR-based Dueling Double Deep Q-Network with Prioritized Experience Replay) algorithm was constructed. This algorithm first reconstructs TSN state and traffic features using a multi-head graph attention residual network (MGAR), and then decodes them using a multi-layer perceptron. PER-MD3QN leverages the MGAR attention mechanism to flexibly adjust weights, selecting information critical to the current task from the TSN and traffic features of each input, ultimately finding a route for the TS stream that ensures a legitimate injection time slot.

[0155] The details are explained below.

[0156] (1) System architecture:

[0157] Figure 2 This is a diagram of a fault-tolerant system architecture for a large-scale TSN network in an optional embodiment of the present invention. Figure 2 The scheduling mechanism is based on SDN and has a system architecture that includes the transport layer, control layer, and application layer. The details are as follows:

[0158] Transport layer: Network elements such as switches, full-duplex physical links, and various IoT devices comprise the transport layer. At the transport layer, user terminals such as drones and VR devices continuously interact with servers, generating a large number of time-sensitive flows. Each TSN switch collects network resource status data at a specified frequency during each transmission cycle and forwards it to the control layer. At the end of the transmission cycle, it reports on its own node operating status and link detection status. The transmission requests generated by the terminals are uploaded to the SDN controller's centralized user configuration controller (CUC). The CUC analyzes the user transmission requests and transfers them to the centralized network configuration (CNC) for processing and distribution.

[0159] Control layer: CNC can be abstracted into functional modules such as monitoring module, analysis module, decision module and scheduling module. The specific functions of each module are as follows:

[0160] Monitoring module: Node and link monitoring, which monitors and receives network status information uploaded by transport layer switches at a certain frequency through southbound interfaces, records links where traffic loss is reduced and problem nodes that fail to report information in a timely manner.

[0161] Analysis module: Receives and records user transmission requests from CUC and network status information provided by the monitoring module. It determines the order of traffic processed by the decision module based on attributes such as whether the traffic is abnormal, arrival time, service priority, and traffic cycle size. When an abnormal situation occurs, it notifies the scheduling module to release the resources occupied by the relevant traffic and give priority to the relevant traffic.

[0162] The decision module receives the transmission tasks and TSN status provided by the analysis module, builds an optimization model for the joint routing and scheduling problem, and uses the PER-MD3QN algorithm to solve the routes with legal injection time slots on a hop-by-hop basis. The result is then passed to the scheduling module. The decision module then collects relevant experience and stores it in a prioritized experience replay pool. It then extracts experience from this pool at a regular interval to train and update the PER-MD3QN network.

[0163] Scheduling module: Receives and executes the operation of releasing traffic occupied resources from the analysis module, checks whether the decision given by the decision module meets the relevant constraints such as traffic delay and network resources, and finds the injection time slot that minimizes the maximum load among the legal decisions. Finally, it modifies the flow table to make the joint routing and scheduling operations effective.

[0164] (2) Network model:

[0165] like Figure 2 As shown, the overall network topology of the transport layer is expressed as G = {V, E}, which represents the overall network topology of the transport layer, with a set of switching nodes V and a set of transmission links E. a ,a∈N} represents a=1,2,3,...,N switching nodes v in the corresponding topology a The set of N switching nodes in the corresponding topology; E = {e a,b |a,b∈N,a≠b} represents the set of physical links ea,b between nodes a and b, and the weight ebwa,b represents the link e a,b The current remaining bandwidth. In addition, use A=R N×N Represents the adjacency matrix of the network topology. a,b =1 represents e a,b ∈E, when A a,b =0 means The SDN controller will update the adjacency matrix A according to the real-time network status. The upper limit of each node port is P, M = R N×P Represents the adjacency table of neighbor node identifiers corresponding to each port of each node in G, m a,p Represents the neighbor of node a corresponding to port p, where

[0166] Assume there are FL transmission tasks. In this embodiment of the present invention, the i-th transmission task is abstracted as flowi , represented as a tuple flow i ={src i ,dst i ,pr i ,d i ,t i ,bw i ,prd i ,φ i The number of streams successfully transmitted at time t is sf t The set of all transfer tasks is F = {flow i ∣i∈FL}. Among them, src i ,ds i t∈V are the source node and destination node identifiers of the transmission. i ,d i ,t i ,bw i ,prd i φ i They represent flow i priority, maximum delay, start time, required bandwidth, sending cycle, and planned transmission path. i Number of priority types ρ,pr i The larger the value, the higher the priority. The scheduling mechanism strictly schedules multiple scheduling requests uploaded at the same time in order of priority; for flows of the same priority, the first-come, first-served principle is applied.

[0167] In order to unify the different transmission periods of time-sensitive flows, the gating cycle period T is equal to the least common multiple of all time-sensitive flow periods, as shown in formula (1). The hardware scheduling time slot sl of CQF queue flipping is shown in formula (2).

[0168] T=lcm(prd i ),i∈FL (1)

[0169] sl=gcd(prd i ),i∈FL (2)

[0170] Assume that the transmission rate of each link is unified as s, and the maximum data frame length is MTU. To avoid congestion, it is necessary to stipulate the maximum value ξ (i.e., the upper limit of the port queue capacity) that the total number of data frames sent by a port cannot exceed, as shown in formula (3). Among them, s is the unified transmission rate of each link, and MTU is the maximum data frame length. trans ,d sync They are propagation delay and synchronization delay respectively.

[0171]

[0172] The upper limit of each transmission node port is P. The CQF queue of each port is virtualized by the SDN controller to obtain ρ different priority sending queues. In each cycle, each queue sends data in descending order of priority. The priority of node a port p is pr. The sending queue usage capacity is expressed as Resource utilization of queue p at node a at time t Node resource utilization and overall network resource utilization As shown in formula (3). Set the high load threshold to thd, when the queue resource utilization When the link corresponding to port p of node a is a high-load link, the total number is recorded as hll t (highload link), while limiting the queue resource usage of normal traffic that exceeds the thd part.

[0173]

[0174] Therefore, the node resource usage status such as queue capacity and link bandwidth of neighboring nodes to themselves can be expressed as The state of each scheduled flow (sfs, scheduled flow state) on the node can be described according to the destination node, priority, data frame and occupied bandwidth recorded in the flow table as follows: Similarly, the processing scheduled flow state (pfs) at this node can be described as where t j Indicates the arrival time at the node. Finally, in order to prevent link loops and traffic from being routed to faulty nodes, Indicates that node a is a faulty node or flow i The transmission path already exists for the node. It represents the shortest hop count from node a to the destination node. Therefore, the overall real-time state of the network is shown in formula (4).

[0175]

[0176] First, collect the flow information related to the problem node link i ={id i ,n src,i ,n dst,i ,pr i ,d i ,t i ,bw i ,prd i ,frm i}; Secondly, modify the flow table and release the node queue capacity occupied by the affected flow and link bandwidth ebw a,b etc. Network resources, correct the relevant representation ns a , sfs, pfs; Then, modify the network topology information and set the link status of each faulty node in the network: Taking node a as an example, its resource utilization is the highest, the capacity of all port queues has reached the upper limit, and the remaining bandwidth of all links pointing to itself is 0, so the status ns a Indicates setting to u a =1, ebw b,a =0, at the same time Setting it to 1 indicates a fault state. Finally, to reduce the impact of the fault, flows that need to be rerouted will take precedence over normal transmission flow scheduling, and the network status and flow information will be uploaded to the problem optimization model in descending order of priority. The model will receive the dispatched scheduling actions and execute the actions to generate the corresponding flow table after all constraints are checked.

[0177] (3) Multi-head graph attention residual network:

[0178] Figure 3 is a schematic diagram of a PER-MD3QN network provided according to an embodiment of the present invention, such as Figure 3 As shown, the MGAR-based encoder model in PER-MD3QN processes the uploaded TSN network topology A and TSN scheduling information consisting of N node features. rts t Indicates the overall current state of the network at time slot t. It is then decoded by the dueling architecture decoder. t The feature extraction will be carried out in sequence through the K-layer MGB (multi-head graphattention networks based block) of the encoder residual connection, and then the obtained link vectors will be spliced ​​into a 1-dimensional vector and provided to the decoder of the duel network architecture for decoding output. Each layer of MGB in the encoder has H independent attention mechanisms, and each attention mechanism can be described as Operation, represents the h-th image attention mechanism in the k-th layer MG B in the multi-head image attention residual network, Represents the feature matrix of the k-th layer MGB input at time slot t, which can be calculated based on the current k-th layer input The traffic scheduling status shown is adaptively adjusting the weight of each neighbor link information, selectively focusing on key information, and performing weighted fusion of information transmission between different links to obtain the output Specifically, Figure 4is a schematic diagram of a graph attention coefficient provided according to an embodiment of the present invention, The calculation is as follows Figure 4 As shown, is the representation of node i in the feature matrix, that is represents the feature of the i-th node input to the k-th layer MGB at time slot t, is the shared linear transformation matrix in the attention mechanism h, ak k.h Node i and node j will pass through High-dimensional features after transformation and splicing is mapped to a real matrix, Is an activation function. As shown in formulas (7) and (8), the attention mechanism h of the kth layer calculates the attention coefficient of each link i and the adjacent chain nodes after normalization. After that, the weighted sum is obtained to get the final representation Finally, the output of the k-th layer MGB is the average of the results of each attention As shown in formula (9), the input of the first layer MGB of the encoder model is rts t , the k+1th layer MGB input It is the sum of the output of the k-th layer MGB and the input obtained by the residual connection of the k-th layer MGB. Therefore, the encoder model input is rts t The output is represents the normalized attention coefficient of link node i and adjacent chain nodes, The feature matrix representing the output of the h-th attention mechanism in the k-th layer MGB at time slot t is based on The result of weighted summation is Represents the k+1th layer MGB input, which is the output of the kth layer MGB and the kth layer MGB input obtained by residual connection sum.

[0179]

[0180]

[0181]

[0182] (4) Problem optimization model:

[0183] 1) Routing hop count constraint:

[0184] Since in the CQF model, flow i End-to-end (E2E) delay iGenerally, it only depends on the range of cycle size sl and path hop number H [(H-1)*sl, (H+1)*sl]. So considering the worst case, flow i At most, H i After jumping, it reaches the destination node. i Indicates flow i The number of hops remaining from the current node to the destination node.

[0185]

[0186] 2) Path constraints:

[0187] If flow i The currently determined path is φ ending at node a i , that is, φ i Indicates flow i The transmission path has been determined. Then, as shown in the constraint of formula (9), each node b in the optional next node set next should have a link pointed to by node a, and the remaining bandwidth capacity of the link should not be less than bw i At the same time, the selected node should not be a faulty node or an invalid node that forms a link loop, and the number of hops from the node to the destination node should not exceed the currently allowed limit.

[0188]

[0189] 3) Optional queue and capacity constraints:

[0190] where v b The corresponding port number p, then ξ deducts the occupied cache capacity There should be flow after i The sending queue buffer space in this time slot is shown in formula (10).

[0191]

[0192] Finally, because the scheduling mechanism of the embodiment of the present invention needs to ensure the scheduling performance of the scheduling mechanism under abnormal scenarios, the scheduling success rate successRa and the number of high-load links hl t l is used as an evaluation factor. The goal is to ensure the load balance of scheduling tasks and reduce the number of high-load links while improving successRa. t , thereby reducing the number of flows that need to be scheduled for retransmission when node failures or link interruptions occur and reserving network resources for scheduling retransmission. Therefore, the problem optimization model can be formally expressed as:

[0193]

[0194] (6) PER-MD3QN algorithm:

[0195] In the scheduling algorithm based on PER-MD3QN solution in the embodiment of the present invention, the specific meanings of state, action, and reward are as follows.

[0196] 1) Status:

[0197] In this scheduling mechanism, the state is the resource information and traffic information of each TSN node collected by the system at each time slot t. Therefore, the environmental state at time t is expressed as shown in formula (12). t Represents the input state of the PER-MD3QN algorithm

[0198] S t =rts t (12)

[0199] 2) Action:

[0200] In this scheduling mechanism, since the direct selection of flow paths will make the action space dimension of the calculation process too high, the model will make full use of real-time network information and flow information to output the appropriate route for each flow and send it to the corresponding sending queue priority, so as to maximize the use of network resources and forward the flow more reasonably. As shown in formula (13), the action at time t is represented by act t , is a set of N*prMax long one-hot vectors. n,pr =1 means that the sending queue pr of node a pointing to node n is selected as flow i The scheduling action is 0. t Represents the action space of the PER-MD3QN algorithm, which is a set of N*prMax long one-hot vectors, a n,pr Indicates the scheduling action for the send queue pr of node n.

[0201]

[0202] 3) Rewards:

[0203] As shown in formula (18), in this scheduling mechanism, the reward function is composed of the reward value for successful scheduling, the reward for traffic scheduled transmission in the intermediate link, and the penalty value for scheduling failure. Among them, the reward for the intermediate link needs to ensure the load balance of network resources while maximizing the scheduling success rate and limiting the occupation of queue resources exceeding the high load threshold. Therefore, it is related to the next link selected. and overall network resource utilization The coefficient λ is a negative number. When the scheduling is successful or the illegal constraint fails, the flow scheduling end flag is done. t will be 1, otherwise it will be 0. represents the reward function of the PER-MD3QN algorithm. λ represents the reward coefficient of the reward function at the intermediate node scheduling. Z represents the reward value as a positive number in the reward function.

[0204]

[0205] 4) Loss function:

[0206] Q-learning is a classic reinforcement learning algorithm whose core idea is to guide the agent to make optimal actions in different states by learning a value function (Q function). In this embodiment of the present invention, the PER-MD3QN algorithm is used to solve the problem optimization model. Figure 5 : is a flow chart of the PER-MD3QN algorithm provided according to an embodiment of the present invention, such as Figure 5 As shown in the figure, the PER-MD3QN algorithm improves the performance of the Q learning algorithm in reinforcement learning tasks by improving the Dueling architecture and Double Q-learning. As shown in formula (19), for the encoder model processing state S t Output The encoder model of the Dueling architecture converts the Q value function Decomposed into state value function (Value function) and advantage function This allows the agent to simultaneously learn the value of the state and the pros and cons of different actions. t represents the network parameters at time t. ) represents the network in the state space Action space act t , network parameters θ t The Q-value function under . Represented in state space Action space act t , network parameters θ t The state value function (Value function) under . Represented in state space Action space act t , network parameters θ t Advantage function under .

[0207]

[0208] Q learning uses the temporal-difference error ζ in experience replay t The learning update value function is updated as shown in Equation (20), where δ∈(0,1] represents the learning rate.

[0209]

[0210] TD is the calculation of the difference between the value function estimate of the current state and the value function estimate of the next state. Since the Double Q-learning computing architecture of the PER-MD3QN algorithm has an evaluation network evalQ t (S t ,act t θ t,eval ) and target network targetQ t (S t ,act t θ t,target ),θ t,eval Indicates the network parameters corresponding to the evaluation network. So the estimated value y t and

[0211] The TD calculation method is shown in formula (21) (22). t Indicates whether the current scheduling is finally completed. t Represents PER-MD3QN's estimate of the value function of the current state.

[0212] y t =r t +(1-done t )*γ*targetQ t (S t ′,argmax(evalQ t (S t ′,act t ';θ t,eval ));θ t,target ) (twenty one)

[0213] ζ t =y t -evalQ t (S t ,act t θ t,eval ) (twenty two)

[0214] ζ t The temporal-difference error is the calculation of the difference between the value function estimate of the current state and the value function estimate of the next state. The larger the value, the more the experience needs to be updated. Therefore, the priority experience playback is based on ζ t Give the corresponding experience playback priority per t , as shown in formula (23). Where υ is a very small constant to prevent ζ tThe probability of extracting an experience of 0 is 0. The corresponding sampling probability is shown in formula (24), where the exponent α determines the high and low priority usage, and 0 corresponds to the uniform situation. The priority replay method changes the sampling method of experience, but it introduces bias, which is not conducive to the training of neural networks. As shown in formula (27), by using importance sampling To correct the error, where the exponent β indicates the degree of correction. t represents the sampling probability corresponding to the experience replay priority, υ represents a small constant to prevent ζ t The probability of 0 experience being extracted is 0. α represents the priority per t The degree of use of , where 0 corresponds to the uniform situation. represents importance sampling, which is used to correct the priority error introduced. β represents the degree of correction.

[0215] per t =|ζ t |+υ (23)

[0216] PER t =(per t ) α ∑(per t ) α (twenty four)

[0217] w t =(per t min(per t )) β (25)

[0218] Therefore, the PER-MD3QN algorithm learns the optimal network parameters θ by iteratively minimizing the loss function t , the loss function is shown in formula (26), where L(θ t,eval ) represents the minimization loss function, which is used to learn the optimal network parameters θ t :

[0219]

[0220] The loss function can be used to optimize the network using stochastic gradient descent, and its gradient calculation is shown in formula (27), where, represents the stochastic gradient descent used to optimize the network, and τ represents the coefficient for soft updating of the parameters of the evaluation network:

[0221]

[0222] 5) Network Update:

[0223] The PER-MD3QN algorithm updates parameters by calculating stochastic gradient descent using a small batch of weighted samples from the experience replay pool ReplyBuffer, as shown in Equation (28). At the same time, after several updates, the target network is soft-updated with a certain proportion of the evaluation network's parameters, as shown in Equation (29).

[0224]

[0225] θ t,target ←τ*θ t,eval +(1-τ)*θ t,target (29)

[0226] Algorithm 1: PER-MD3QN algorithm:

[0227] The algorithm inputs the transmission domain network topology G = {V, E}, the priority experience replay pool capacity M, the set of transmission tasks requested for scheduling F, the learning rate δ, the soft update coefficient τ, the discount factor γ, the number of training rounds ep, the number of training steps sp, and outputs the action Act for each scheduling request.

[0228]

[0229] The algorithm randomly initializes the parameters of the evaluation network evalQ in the transmission domain G, and copies the evaluation network parameters to the target network targetQ, and initializes the priority replay pool RB (lines 1-2).

[0230] Then process the status. Get the TSN status rts at time t t , process to get state S t (line 5).

[0231] Execute the action. Select the action act using the ε-greedy algorithm t (Lines 6-7). Execute the action act t And according to formula (18), the reward value r is obtained t , get the end mark according to the actual situation and TSN new status rts t '(line 8). After that, the next flow state S is processed. t ′, and store the experience (S t ,act t ,r t ,S t ′, done) in the experience replay pool (9th to 10th lines). According to the end mark Update the flow that needs to be scheduled (line 11).

[0232] Update the network. Take some experience from the experience pool to train and evaluate the network, and calculate the gradient descent using formulas (27)(28)(29) And update the evaluation network, and every sp step, use the evaluation network parameter θ in proportion to the soft update coefficient τ t,eval Update the target network parameters θ t,target (Lines 12 to 15).

[0233] Based on the above embodiments, the following effects can be achieved:

[0234] (A) Encoding and decoding model based on multi-head graph attention residual network: This paper designs an encoder based on multi-head graph attention residual network to reconstruct the features of TSN and traffic, and adopts an encoding and decoding model with duel network decoding, which can better adapt to the changing network traffic and extract key information.

[0235] (B) Scheduling mechanism based on the combination of SDN and the D3QN algorithm with a multi-head graph attention residual network: In normal scenarios, it provides incremental scheduling for TS flows online, achieving better scheduling success rates and load balancing effects. When TSN anomalies occur, it can quickly adapt to network changes and reroute faulty TS flows, achieving better fault tolerance than the redundant path approach.

[0236] The following simulation is conducted on the transmission task scheduling scheme of the above computing power network, and the simulation results are presented.

[0237] This simulation environment uses the Python 3.9 NetworkX and SimPy modules to build a network topology and simulate traffic flow. The problem optimization model relies on the multi-head graph attention mechanism-based feature extraction algorithm and the D3QN-based joint routing and queue scheduling algorithm, both built using the Pytorch library. In the problem optimization model, the number of MGB layers K is set to 2, the multi-head attention mechanism H per layer is set to 3, the reward function constant Z is set to 1.5, and the parameter λ is set to -0.25. The model is trained to improve the scheduling success rate while reducing average resource utilization. The high-load link threshold thd is set to 0.9. This embodiment of the present invention randomly generates a network scenario consisting of 10 terminal nodes and 90 transmission nodes. The link latency is 1-2 ms, and the initial link capacity is set to 350 Mbit / s. The super-period slot T for cyclic forwarding on each node is 10 ms, and the buffer capacity of each queue is 100 data frames. During a 30-second run, 200, 300, and 500 time-sensitive flows with random starting and destination nodes were sent at random times, ranked by priority from high to low, to simulate network uncertainty. Each flow's bandwidth requirement followed a Poisson distribution, with a transmission period of 2ms, 5ms, or 10ms. The maximum latency was related to the average end-to-end latency of the corresponding path. The algorithm training parameters are shown in Table 1.

[0238] Table 1

[0239]

[0240] Results and Analysis:

[0241] In this simulation, to evaluate the scheduling performance of the SRGD scheduling mechanism, SRGD was combined with a heuristic-based dynamic scheduling and routing algorithm (HDSR), a heuristic-based shared fault-tolerant segment algorithm (HSFS), a deep-inforcement learning-based scheduler (DRLS), and a scheduler based on DRL with integrated GCN (DIGS). HDSR considers the impact of source node location on the overall routing schedule and generates multiple disjoint paths to transmit redundant data. HSFS maximizes service quality with minimal bandwidth by setting up shared fault-tolerant network segments for different critical flows. DRLS perceives topology changes by recording the shortest path matrix and guides the DRL algorithm to output scheduling actions consisting of links and time slots. DIGS, based on GCN, perceives network resources in the graph structure and outputs scheduling actions with the goal of minimizing average latency. The indicators for scheduling performance evaluation are the scheduling success rate successRate and the total number of high-load links hll t .

[0242] Algorithm convergence:

[0243] In a large-scale network environment, the centralized scheduling of the problem optimization model is difficult to produce effective actions, resulting in an excessively high failure rate in the initial stage of training, which cannot provide the model with sufficient successful experience for training. In order to accelerate the training process, the embodiment of the present invention adopts two strategies: 1) Applying a control gate mechanism to ignore invalid actions that violate constraints 2 and 3 with a certain probability, thereby increasing the occurrence of successful experience; 2) Using a prioritized experience replay method, the model pays more attention to samples that have an important impact on improving model performance, thereby accelerating the training convergence process and improving learning efficiency; 3) The duel network is used in the PER-MD3QN algorithm to simultaneously learn the value of the state and the pros and cons of different actions, while separating the evaluation network and the target network to reduce the problem of over-estimation and thus improve the stability and performance of the algorithm.

[0244] Figure 6 is a schematic diagram showing the change of average reward with the number of training steps during the simulation process of an embodiment of the present invention, such as Figure 6As shown, the average reward results of the algorithm after 50,000 training steps at different learning rates generally show a gradual upward trend with the increase in training steps. At the same time, the oscillation of the curve gradually decreases as the strategy is optimized. Specifically, when the learning rate is 1e-3, the average reward curve converges at around 20,000 steps; when the learning rate is 2e-4, the curve converges at around 40,000 steps; and when the learning rate is 5e-4, the convergence rate is somewhere in between. Although different learning rates have a certain impact on convergence, it can be seen that the reward values ​​obtained after convergence at each learning rate are relatively small and remain within a relatively stable range. This shows that the learning rate between 2e-4 and 1e-3 has a certain impact on the convergence rate of the algorithm of the present invention, but has a relatively small impact on performance.

[0245] Impact of traffic scale on scheduling performance in normal scenarios:

[0246] Figure 7 FIG. 1 is a schematic diagram showing how the scheduling success rate changes with the number of time-sensitive flows during the simulation process of an embodiment of the present invention. Figure 7 As shown in the figure, as network traffic increases, the scheduling success rate of each strategy generally decreases, but the SRGD strategy generally has a higher scheduling success rate than other strategies. This is because SRGD uses a coding model based on the GAT residual network, which, compared to strategies other than DIGS, can better perceive resource changes at each node in the graph-structured network, thereby ensuring scheduling success while maintaining resource load balancing. DIGS, on the other hand, prioritizes minimizing average latency when scheduling flows and pays less attention to resource load balancing. Its scheduling success rate is lower than that of SRGD when traffic scales are large. HDSR's redundant path fault tolerance requires more resources for each flow to be successfully scheduled, resulting in the lowest scheduling success rate under limited resources. HSFS, on the other hand, uses a shared fault-tolerant segment fault tolerance method that is more resource-efficient than HDSR, resulting in a relatively high scheduling success rate.

[0247] Figure 8 FIG. 1 is a schematic diagram showing the change in the number of high-load links with the number of time-sensitive flows during the simulation process of an embodiment of the present invention. Figure 8 As shown, the hll of each strategy t As the scale of network traffic increases, the number of scheduled successful flows increases. Among them, SRGD ensures the highest scheduling success rate among all strategies while making hll t At least, HDSR has a lower scheduling success rate than DRLS and DIGS due to the planning of multiple redundant paths for the flow. t However, it is higher than the latter two, so the network utilization of HDSR is low and has a negative impact on the scheduling and routing of subsequent flows.

[0248] Impact of the number of abnormal nodes on scheduling performance:

[0249] To simulate abnormal node failures and link disruptions, after all traffic is scheduled in a normal scenario, random failures occur at 3, 6, and 12 nodes out of 100 network nodes, leading to corresponding link disruptions. Each strategy reschedules the flows associated with the failures. Because DIGS uses GCN, which is significantly affected by network topology, similar to SRGD, it only modifies the state of network node links, leaving the existing network topology unchanged.

[0250] Figure 9 FIG. 1 is a schematic diagram showing the change of the scheduling success rate under different numbers of fault nodes during the simulation process of an embodiment of the present invention. Figure 9 As shown, the scheduling success rates of various strategies decline to varying degrees under abnormal scenarios. SRG D, because its GAT can distinguish between abnormal and normal nodes, better adapts to TSN in abnormal scenarios, achieving the best scheduling success rate compared to other strategies. HDSR, relying on redundant path fault tolerance, shows a slight decrease in scheduling success rate when six nodes fail compared to normal conditions. However, with 12 failed nodes, fault tolerance fails due to the interruption of redundant paths for multiple flows. Furthermore, HDSR's high scheduling cost significantly reduces the scheduling success rate. HSFS has limited shared fault tolerance segments, so the scheduling success rate drops significantly when the number of faulty nodes increases and the shared fault tolerance segments are exhausted. DRLS, due to its poor graph-based network perception, experiences a significant drop in scheduling success rate after topology changes. Although DIGS's scheduling performance degrades significantly due to the lack of topology changes and the modification of the Laplacian matrix, the node aggregation process convolves the features of abnormal nodes, significantly reducing its scheduling success rate.

[0251] Figure 10 FIG. 1 is a schematic diagram showing the change in the number of high-load links under different numbers of fault nodes during the simulation process of an embodiment of the present invention. Figure 10 As shown, with the increase of fault nodes, each strategy hll t And except for SRGD, each strategy hll t The increase shows a trend of increasing first and then decreasing. This is because when there are fewer faulty nodes, the success rate of each strategy for rerouting interrupted flows is high, which increases the load on the remaining links too much. When there are more faulty nodes, fewer interrupted flows are successfully rescheduled, which reduces the increase in high-load links. However, because SRGD relies on GAT to perceive changes in the resource status of each node, it can flexibly schedule traffic using existing resources in abnormal scenarios to improve the scheduling success rate. At the same time, it balances the load of network resources as much as possible to accommodate more traffic. Therefore, the success rate of rescheduling interrupted flows remains high, hll t The rate of increase shows an increasing trend.

[0252] Impact of the number of MGB layers on scheduling performance:

[0253] Through the above experimental analysis, we can see that SRGD relies on the coding model based on the GAT residual network to perceive the network node status and thus achieve flexible traffic routing scheduling, and the number of MGB layers will affect the feature extraction ability of the coding model. Therefore, in order to further explore the impact of the number of MGB layers on the algorithm performance, Figure 11 FIG. 1 is a schematic diagram showing the change of the scheduling success rate under different numbers of MGBs during the simulation process of an embodiment of the present invention. Figure 11 As shown in Figure 2, the performance of SRGD is explored under the conditions of 0, 3, 6, and 12 node failures.

[0254] Figure 12 FIG. 1 is a schematic diagram showing the change in the number of high-load links under different numbers of MGBs during the simulation process of an embodiment of the present invention. Figure 12 As shown in the figure, although residual connections are added to MGB, the over-smoothing problem cannot be completely eliminated. When the number of MGB layers increases, the information propagation in the multi-layer GAT will make the representation of the nodes tend to cluster together, resulting in a decrease in the differentiation between nodes, which in turn affects the performance of the model. Therefore, the scheduling success of SRGD first increases with the increase of the number of MGB layers, and the best effect is achieved when the number of MGB layers is 2, and then decreases with the increase of the number of MGB layers. The figure shows the SRGD network resource utilization with different MGB layers. When the number of MGB layers is greater than 2, the SRGD scheduling success rate is small, while hll t This is because the over-smoothing problem affects scheduling performance, resulting in the inability to select appropriate nodes when scheduling traffic. As a result, the transmission path for traffic distribution is long, occupying unnecessary network resources, or the distribution link load is too high.

[0255] In an embodiment of the present invention, a device for scheduling transmission tasks of a computing power network is also provided. Figure 13 is a structural block diagram of a scheduling device for transmission tasks of a computing power network provided in an embodiment of the present invention, such as Figure 13 As shown, the device includes: a first acquisition module 132, a second acquisition module 134 and a processing module 136. The device is described below.

[0256] The first acquisition module 132 is used to obtain the network status of the computing power network based on the software-defined network SDN, wherein the network status includes the status of the transmission nodes and the status of the transmission links in the computing power network, and the computing power network includes faulty transmission nodes and / or faulty transmission links; the second acquisition module 134 is connected to the above-mentioned first acquisition module 132, and is used to obtain the target transmission task of the computing power network; the processing module 136 is connected to the above-mentioned second acquisition module 134, and is used to input the network status and target transmission task into the target network model to obtain the scheduling route of the target transmission task, wherein the target network model reconstructs the node status characteristics of the transmission nodes and the link flow characteristics of the transmission links in the computing power network based on the multi-head graph attention residual network, and adopts the attention mechanism to adjust the weights of the node status characteristics and the link flow characteristics; based on the adjusted weights of the node status characteristics and the link flow characteristics, a duel network is adopted to select the transmission link for the target transmission task, wherein the multiple transmission links obtained hop by hop based on the multi-head graph attention residual network and the duel network constitute the scheduling route.

[0257] In an embodiment of the present invention, a computer device is also provided, including: a memory and a processor, the memory storing a computer program; the processor, for executing the computer program stored in the memory, and a method for scheduling transmission tasks of a computing power network in which the processor executes any one of the above-mentioned tasks when the computer program is running.

[0258] In an embodiment of the present invention, a computer-readable storage medium is also provided, characterized in that the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute a scheduling method for transmission tasks of the computing power network described above.

[0259] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0260] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0261] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0262] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0263] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0264] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0265] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for scheduling transmission tasks of a computing network, characterized in that: include: Obtaining a network status of a computing power network based on a software-defined network (SDN), wherein the network status includes a status of a transmission node and a status of a transmission link in the computing power network, and the computing power network includes a faulty transmission node and / or a faulty transmission link; Obtaining the target transmission task of the computing power network; The network state and the target transmission task are input into a target network model to obtain a scheduling route for the target transmission task, wherein the target network model reconstructs the node state characteristics of the transmission nodes in the computing power network and the link flow characteristics of the transmission link based on a multi-head graph attention residual network, and uses an attention mechanism to adjust the weights of the node state characteristics and the weights of the link flow characteristics; based on the adjusted weights of the node state characteristics and the weights of the link flow characteristics, a duel network is used to select a transmission link for the target transmission task, wherein a plurality of transmission links obtained hop by hop based on the multi-head graph attention residual network and the duel network constitute the scheduling route; Before inputting the network state and the target transmission task into the target network model to obtain the scheduling route of the target transmission task, it also includes: obtaining a sample set for training the initial network model; based on the sample set, using the MGAR-based Prioritized Experience Replay Duel Dual Depth Q Network PER-MD3QN algorithm to train the initial network model to obtain the target network model, wherein the PER-MD3QN algorithm includes three elements: state, action, and reward, wherein the state is the state of the transmission node and the state of the transmission link in the computing power network, and the action is to output a suitable route for each flow according to the state and send it to the corresponding sending queue priority, and the reward is generated by the reward value of successful scheduling, the reward for traffic scheduling and transmission in the intermediate link, and the penalty value of scheduling failure.

2. The method according to claim 1, characterized in that The obtaining of the network status of the computing network based on the software defined network (SDN) includes: Obtaining the topology of the computing power network; Acquire a set of transmission nodes in the topology structure and a set of transmission links in the topology structure; Determining the status of each transmission node in the transmission node set includes at least one of the following: a capacity upper limit of a port of the transmission node, different priority sending queues virtualized by the circular queue and forwarding queue on the port of the transmission node through the SDN; resource utilization of the transmission node, resource utilization of the sending queue, resource utilization of the computing network, status of traffic already scheduled on the transmission node, status of traffic being scheduled on the transmission node, and whether the transmission node is a faulty node; Determining the status of each transmission link in the transmission link set includes at least one of the following: a transmission rate of the transmission link, a maximum data frame length of the transmission link, a propagation delay of the transmission link, a synchronization delay of the transmission link, a load rate of the transmission link, and whether the transmission link is a faulty link.

3. The method according to claim 1, characterized in that The obtaining of the target transmission task of the computing power network includes: Acquire multiple transmission tasks in the computing power network and the priorities of the multiple transmission tasks; selecting a candidate transmission task with the highest priority from the multiple transmission tasks based on the priorities of the multiple transmission tasks; In the case that there are multiple candidate transmission tasks with the highest priority, the candidate transmission task with the earliest task request time is selected as the target transmission task.

4. The method according to claim 3, characterized in that The obtaining the priorities of the multiple transmission tasks includes: It is determined that the priority of the transmission task corresponding to the flow that needs to be rerouted is higher than the priority of the transmission task corresponding to the normal flow.

5. The method according to claim 1, wherein The multi-head graph attention residual network in the target network model includes multiple layers, each of which has multiple independent attention mechanisms. Each attention mechanism is used to adaptively adjust the weight of each neighbor link information according to the traffic scheduling status indicated by the current layer input, selectively focus on key information, and perform weighted fusion of information transmission between different links to obtain output.

6. The method according to claim 1, characterized in that The method of training the initial network model based on the sample set and obtaining the target network model by adopting the MGAR-based Prioritized Experience Replay Duel Dual-Deep Q Network PER-MD3QN algorithm comprises: Based on the sample set, the PER-MD3QN algorithm and the Q-learning training method are used to train the initial network model to obtain the target network model, wherein when the Q-learning training method is used, the Q-value function is decomposed into a state value function and an advantage function, and the state value function is updated based on a time difference error, and the time difference error is the difference between the function value of the current state and the function value of the next state.

7. The method according to claim 1, characterized in that The method of training the initial network model based on the sample set and obtaining the target network model by adopting the MGAR-based Prioritized Experience Replay Duel Dual-Deep Q Network PER-MD3QN algorithm comprises: Based on the sample set, the initial network model is trained using the MGAR-based Prioritized Experience Replay Duel Dual Deep Q Network PER-MD3QN algorithm to obtain an intermediate network model; Determining constraints for optimizing the intermediate network model, wherein the constraints include at least one of: a routing hop count constraint, a path constraint, an optional queue, and a capacity constraint; Based on the constraint conditions, the intermediate network model is optimized to obtain the target network model.

8. A device for scheduling transmission tasks of a computing network, characterized in that: include: A first acquisition module is configured to acquire a network status of a computing power network based on a software-defined network (SDN), wherein the network status includes a status of a transmission node and a status of a transmission link in the computing power network, and the computing power network includes a faulty transmission node and / or a faulty transmission link; A second acquisition module is used to obtain the target transmission task of the computing power network; A processing module is configured to input the network state and the target transmission task into a target network model to obtain a scheduling route for the target transmission task, wherein the target network model reconstructs the node state characteristics of the transmission nodes in the computing network and the link flow characteristics of the transmission link based on a multi-head graph attention residual network, and uses an attention mechanism to adjust the weights of the node state characteristics and the link flow characteristics; based on the adjusted weights of the node state characteristics and the link flow characteristics, a duel network is used to select a transmission link for the target transmission task, wherein a plurality of transmission links obtained hop by hop based on the multi-head graph attention residual network and the duel network constitute the scheduling route; The device is further used to obtain a sample set for training the initial network model before inputting the network state and the target transmission task into the target network model to obtain the scheduling route of the target transmission task; based on the sample set, the MGAR-based priority experience replay duel dual-depth q network PER-MD3QN algorithm is used to train the initial network model to obtain the target network model, wherein the PER-MD3QN algorithm includes three elements: state, action, and reward, wherein the state is the state of the transmission node and the state of the transmission link in the computing power network, and the action is to output a suitable route for each flow according to the state and send it to the corresponding sending queue priority, and the reward is generated by the reward value of successful scheduling, the reward for traffic scheduling and transmission in the intermediate link, and the penalty value of scheduling failure.

9. A computer device, characterized in that: include: memory and processor, The memory stores a computer program; The processor is used to execute the computer program stored in the memory, and when the computer program is running, the processor executes the scheduling method for transmission tasks of the computing power network described in any one of claims 1 to 7.

Images