Routing optimization method and system based on graph structure characteristics

Abstract

The present invention discloses a routing optimization method and system based on graph structure features. The system is used in an SDN network environment, including a control plane and a data plane, where the control plane includes an information acquisition module, a strategy deployment module, and a DGL module. The method acquires network topology structure information and network information, generates corresponding graph adjacency matrices and network information feature matrices, trains a graph learning neural network based on the graph adjacency matrix and network information feature matrix, obtains a deep graph learning model that minimizes SDN network routing overhead and maximizes link utilization, and deploys the deep graph learning model in the SDN network. The method learns dynamic and complex network topologies in spatial dimensions, solves dynamic topology optimization problems, and provides better routing solutions for SDN networks.

Description

[Technical Field]

[0001] The present invention relates to the technical field of computer networks, and more particularly to a routing optimization method and system based on graph structure characteristics. [Background technology]

[0002] In recent years, with the increasing complexity of network environments and the diversification of business traffic, routing optimization has become a hot research topic. Traditional networks employ a best-effort model for routing selection, using OSPF technology to provide the shortest route. However, this approach is unable to adapt to dynamic and complex network environments. The introduction of software-defined network (SDN) architectures separates the control and data planes of traditional networks, significantly expanding the scope for solving routing optimization problems. In SDN environments, combining deep reinforcement learning with neural networks can be extremely useful for routing decisions. However, algorithms such as CNN, RNN, and LSTM are inherently applicable to Euclidean spaces such as images and grids. Network topologies are typically complex models with strong spatial correlations between links and nodes. Traditional neural networks have difficulty representing these features. Therefore, routing optimization models based on deep reinforcement learning require retraining when the network topology changes and lack the ability to generalize to dynamic topologies. Therefore, a method is needed that can extract the spatial features of network topology, learn dynamic and complex network topology in spatial dimensions, solve the dynamic topology optimization problem, and provide better routing solutions. Summary of the Invention [Problem to be solved by the invention]

[0003] The objective of the present invention is to provide a routing optimization method and system based on graph structure characteristics, which is suitable for switches or routing devices to support traditional Layer 2 network protocols in an SDN network environment, realizes optimization of global routing overhead from multiple network attributes, can adapt to dynamic and complex SDN networks, and ensures the performance of SDN networks. [Means for solving the problem]

[0004] To achieve the above functions, the present invention designs a routing optimization method based on graph structure characteristics, which performs the following steps S1 to S3 for the target SDN network, obtains the routing overhead of each link in the target SDN network, and adjusts the weight of each link to complete the routing optimization of the target SDN network.

[0005] Step S1: For the target SDN network, obtain a network topology graph of the target SDN network based on the southbound interface protocol, construct a graph adjacency matrix according to the connection relationship between each node on each link of the target SDN network in the network topology graph, construct an information feature vector of each node for each node on each link of the target SDN network based on the link bandwidth, traffic, packet loss rate, and transmission delay of each node, and construct a network information feature matrix of the target SDN network based on the information feature vector of each node.

[0006] Step S2: Using the graph adjacency matrix and network information feature matrix as the state of the target SDN network, the graph learning neural network is used to input the graph adjacency matrix and network information feature matrix, and the routing policy and routing overhead of the target SDN network in the current state are output using the deep graph learning method. The network parameters of the graph learning neural network are updated using the gradient backpropagation method, and the graph learning neural network is trained after a predetermined number of iterations to obtain a deep graph learning model that minimizes the routing overhead and maximizes the link utilization of the target SDN network.

[0007] Step S3: Using the trained deep graph learning model, obtain a routing policy that minimizes the routing overhead of the target SDN network based on the state of the target SDN network, deploy the routing policy to the target SDN network, and change the weight of each link in the target SDN network according to the routing policy, thereby completing the routing optimization of the target SDN network.

[0008] As a preferred technical solution of the present invention, the specific steps of step S1 are as follows:

[0009] Step S1.1: For the target SDN network, obtain a network topology structure of the target SDN network based on the southbound interface protocol, where the network topology structure includes M routers and N links.

[0010] Step S1.2: In the network topology structure of the target SDN network, each router corresponds to one real node, each link corresponds to one edge, and virtual nodes are inserted into the edges corresponding to each link. The network topology structure of the target SDN network is represented by a network topology graph G(V, E) having M real nodes, N virtual nodes, and 2N edges, where V represents a node set and E represents an edge set. Specifically, it is as shown in the following formula: V={V 実 , V 仮想} where V 実 represents the set of real nodes, and V 仮想 represents the set of virtual nodes, V 実 ={v s1 , v s2 , ..., v sM} where v s1 , v s2 , ..., v sM represents M real nodes, V 仮想 ={v x1 , v x2 , ..., v xN} where v x1 , v x2 , ..., v xN represents N virtual nodes, E = {e1, e2, ..., e 2N} where e1, e2, ..., e 2N represents 2N edges.

[0011] Step S1.3: x=M+N, where x represents the total number of nodes, and the nodes include M real nodes and N virtual nodes. Based on the network topology graph of the target SDN network, construct an x-order graph adjacency matrix A as shown in the following formula:

number

number

[0012] Step S1.4: For any node i in the target SDN network, based on the link bandwidth, traffic, packet loss rate, and transmission delay of node i, the information feature vector h of node i is calculated as follows: i Build h i =[B wi , T hi , L pi , D ti ] In the formula, B wi is the link bandwidth of node i, and T hi is the traffic of node i, and L pi is the packet loss rate of node i, and D ti is the transmission delay of node i, Based on the information feature vector of each node, construct the network information feature matrix H of the target SDN network, as shown in the following formula:

number

[0013] As a preferred technical solution of the present invention, for the node i in step S1.4, if the node i is a virtual node, the traffic T hi , packet loss rate L pi , transmission delay D ti is 0, and if node i is a real node, the link bandwidth B wi is 0.

[0014] As a preferred technical solution of the present invention, the deep graph learning method in step S2 includes four graph learning neural networks and one experience pool, where the four graph learning neural networks are respectively an online graph strategy network, an online graph value network, a target graph strategy network, and a target graph value network, and the four graph learning neural networks respectively include one input layer, two hidden layers, and one output layer.

[0015] The input layer of the online graph strategy network and the target graph strategy network takes the graph adjacency matrix A and the network information feature matrix H as input, and the output of the online graph strategy network and the target graph strategy network becomes the input of the online graph value network and the target graph value network, respectively. Here, the propagation formula from the input layer to the hidden layer and between hidden layers of each graph learning neural network is the same. If the input layer is the 0th layer, the first hidden layer is the 1st layer, and the second hidden layer is the 2nd layer, the propagation formula is as follows:

number

number

number

number

number

number

[0016] As a preferred technical solution of the present invention, the specific steps of step S2 are as follows:

[0017] Step S2.1: Initialize the weight matrices of the online graph strategy network, the online graph value network, the target strategy network, and the target graph value network, where the weight matrix of the online graph strategy network is W θ and the weight matrix of the online graph value network is W θ′ and the weight matrix of the target graph strategy network is W ω and the weight matrix of the target graph value network is W ω′ is.

[0018] Step S2.2: Initialize the experience pool, the specific steps are as follows:

[0019] Step S2.2.1: Let the graph adjacency matrix A and the network information feature matrix H be the state S of the target SDN network, and define S = [A, H]. t represents the state of the target SDN network at time t, and s t =[A t , H t ] and A t is the graph adjacency matrix of the target SDN network at time t, and H t represents the network information feature matrix of the target SDN network at time t.

[0020] Step S2.2.2:

number

number

[0021] Step S2.2.3: Define the experience pool R as follows:

number

number

[0022] Step S2.3: Iterate a predetermined number of times for the target SDN network, where the predetermined number of iterations is T, and the specific steps are as follows:

[0023] Step S2.3.1: As t=1, obtain the initial state s1 of the target SDN network; Step S2.3.2: The online graph strategy network is calculated based on the state s of the target SDN network at time t. t Depending on the routing policy

number

number

number

number

number

number

number

number

number

number

number

number

number

[0024] As a preferred technical solution of the present invention, the specific steps of step S3 are as follows:

[0025] Step S31: Obtain a graph adjacency matrix A and a network information feature matrix H of the target SDN network; Step S32: Based on the trained deep graph learning model, obtain a routing policy that minimizes the routing overhead of the target SDN network according to the state [A, H] of the target SDN network; Step S33: Deploy the routing policy obtained in step S32 to the target SDN network, and change the weight of each link in the target SDN network according to the routing policy; Step S34: In the traffic transmission process, perform traffic transmission using the updated weight of each link according to the shortest path solution.

[0026] The present invention also designs a system for a routing optimization method based on graph structure characteristics, where a target SDN network includes a control plane and a data plane, and the control plane includes an information acquisition module, a strategy deployment module, and a DGL module, thereby realizing the routing optimization method based on graph structure characteristics.

[0027] Each link and each node of the target SDN network is arranged in the data plane, and the information acquisition module of the control plane is used to acquire the network topology graph of the target SDN network, generate a graph adjacency matrix and a network information feature matrix, and send them to the DGL module.

[0028] The DGL module is based on a graph learning neural network, and takes the graph adjacency matrix and the network information feature matrix as input, and uses the deep graph learning method to output the routing overhead of the target SDN network in the current state. The DGL module updates the network parameters of the graph learning neural network based on the gradient backpropagation method, and trains the graph learning neural network after a preset number of iterations to obtain a deep graph learning model that minimizes the routing overhead and maximizes the link utilization of the target SDN network.

[0029] The strategy placement module of the control plane is used to obtain a routing policy that minimizes the routing overhead of the target SDN network based on the state of the target SDN network using the trained deep graph learning model obtained by the DGL module, and send the routing policy and the routing overhead of the target SDN network to the data plane. [Effects of the Invention]

[0030] The beneficial effects are as follows: Compared with the prior art, the present invention includes the following advantages:

[0031] 1. Use graph learning neural networks to capture the spatial relationships between nodes and links in the network topology.

[0032] 2. Use strategy networks and value networks to perform unsupervised learning on the algorithm, making the algorithm's learning capabilities more detailed.

[0033] 3. Use intelligent algorithms to optimize routing overhead in SDN network environments and improve link utilization, thereby optimizing average end-to-end delay, packet loss rate, and throughput.

[0034] 4. Deep graph learning models have strong generalization capabilities. Trained deep graph learning models maintain their effectiveness even when the network topology changes, and can adapt to large, dynamic, and complex networks. [Brief explanation of the drawings]

[0035] [Figure 1] 1 is a block diagram of an overall system for a routing optimization method based on graph structure features according to an embodiment of the present invention; [Figure 2] FIG. 2 is a block diagram of a DGL algorithm according to an embodiment of the present invention. [Figure 3] FIG. 1 is a diagram illustrating a configuration of a graph learning neural network according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0036] The present invention will be further described below with reference to the accompanying drawings. The following examples are provided to more clearly illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention.

[0037] The routing optimization method based on graph structure characteristics according to an embodiment of the present invention performs the following steps S1 to S3 for the target SDN network, obtains the routing overhead of each link in the target SDN network, and adjusts the weight of each link to complete the routing optimization of the target SDN network.

[0038] Step S1: As shown in Figure 1, for the target SDN network, obtain the network topology graph of the target SDN network based on the southbound interface protocol, construct a graph adjacency matrix according to the connection relationship between each node on each link of the target SDN network in the network topology graph, construct an information feature vector of each node for each node on each link of the target SDN network based on the link bandwidth, traffic, packet loss rate, and transmission delay of each node, and construct a network information feature matrix of the target SDN network based on the information feature vector of each node.

[0039] The specific steps of step S1 are as follows.

[0040] Step S1.1: For the target SDN network, obtain a network topology structure of the target SDN network based on the southbound interface protocol, where the network topology structure includes M routers and N links.

[0041] Step S1.2: In the network topology structure of the target SDN network, each router corresponds to one real node, each link corresponds to one edge, and virtual nodes are inserted into the edges corresponding to each link. The network topology structure of the target SDN network is represented by a network topology graph G(V, E) having M real nodes, N virtual nodes, and 2N edges, where V represents a node set and E represents an edge set. Specifically, it is as shown in the following formula: V={V 実 , V 仮想} where V 実 represents the set of real nodes, and V 仮想 represents the set of virtual nodes, V 実 ={v s1 , v s2 , ..., v sM} where v s1 , v s2 , ..., v sM represents M real nodes, V 仮想 ={v x1 , v x2 , ..., v xN} where v x1 , v x2 , ..., v xN represents N virtual nodes, E = {e1, e2, ..., e 2N} where e1, e2, ..., e 2N represents 2N edges.

[0042] Step S1.3: x=M+N, where x represents the total number of nodes, and the nodes include M real nodes and N virtual nodes. Based on the network topology graph of the target SDN network, construct an x-order graph adjacency matrix A as shown in the following formula:

number

number

[0043] Step S1.4: For any node i in the target SDN network, based on the link bandwidth, traffic, packet loss rate, and transmission delay of node i, the information feature vector h of node i is calculated as follows: i Build h i =[B wi , T hi , L pi , D ti ] In the formula, B wi is the link bandwidth of node i, and T hi is the traffic of node i, and L pi is the packet loss rate of node i, and D ti is the transmission delay of node i.

[0044] Regarding the node i, if the node i is a virtual node, the traffic T hi , packet loss rate L pi , transmission delay D ti is 0, and if node i is a real node, the link bandwidth B wi is 0.

[0045] Based on the information feature vector of each node, construct the network information feature matrix H of the target SDN network, as shown in the following formula:

number

[0046] Step S2: Using the graph adjacency matrix and network information feature matrix as the state of the target SDN network, the graph learning neural network uses the graph adjacency matrix and network information feature matrix as input, and the routing policy and routing overhead of the target SDN network in the current state as output using a deep graph learning method (DGL). The network parameters of the graph learning neural network are updated based on the gradient backpropagation method, and the graph learning neural network is trained after a preset number of iterations to obtain a deep graph learning model that minimizes the routing overhead and maximizes the link utilization of the target SDN network.

[0047] The deep graph learning method in step S2 includes four graph learning neural networks and one experience pool. As shown in FIG. 2, the four graph learning neural networks are an Online Graph Strategy Network (OGSN), an Online Graph Value Network (OGVN), a Target Graph Strategy Network (TGSN), and a Target Graph Value Network (TGVN), respectively. As shown in FIG. 3, the four graph learning neural networks each include one input layer, two hidden layers, and one output layer.

[0048] The input layer of the online graph strategy network and the target graph strategy network takes the graph adjacency matrix A and the network information feature matrix H as input, and the output of the online graph strategy network and the target graph strategy network becomes the input of the online graph value network and the target graph value network, respectively. Here, the propagation formula from the input layer to the hidden layer and between hidden layers of each graph learning neural network is the same. If the input layer is the 0th layer, the first hidden layer is the 1st layer, and the second hidden layer is the 2nd layer, the propagation formula is as follows:

number

number

number

number

[0049] In the online graph value network and the target graph value network, W 1 and W 2 are all 1x1 matrices, the output layer is an aggregation layer, and its output value is a 1x1 matrix, denoted as Value, and is specifically as shown in the following formula:

number

number

[0050] As shown in FIG. 2, the specific steps of step S2 are as follows:

[0051] Step S2.1: Initialize the weight matrices of the online graph strategy network, the online graph value network, the target strategy network, and the target graph value network, where the weight matrix of the online graph strategy network is W θ and the weight matrix of the online graph value network is W θ′ and the weight matrix of the target graph strategy network is W ω and the weight matrix of the target graph value network is W ω′ At the time of initialization, the network parameters of the online graph strategy network and the target strategy network are consistent, and the network parameters of the online graph value network and the target graph value network are consistent.

[0052] Step S2.2: Initialize the experience pool, the specific steps are as follows:

[0053] Step S2.2.1: Let the graph adjacency matrix A and the network information feature matrix H be the state S of the target SDN network, and define S = [A, H]. t represents the state of the target SDN network at time t, and s t =[A t , H t ] and A t is the graph adjacency matrix of the target SDN network at time t, and H t represents the network information feature matrix of the target SDN network at time t.

[0054] Step S2.2.2:

number

number

[0055] The target function U that maximizes the utilization of the target SDN network links. max (B w , T h , L p , D t ) to build a

[0056] Step S2.2.3: Define the experience pool R as follows:

number

number

[0057] Step S2.3: Iterate a predetermined number of times for the target SDN network, where the predetermined number of iterations is T, and the specific steps are as follows:

[0058] Step S2.3.1: As t=1, obtain the initial state s1 of the target SDN network; Step S2.3.2: The online graph strategy network is calculated based on the state s of the target SDN network at time t. t Depending on the routing policy

number

number

number

number

number

number

number

number

number

number

[0059] Step S2.3.8: The loss of the output value of the online graph value network is calculated using the following formula: ogvn Calculate

number

[0060] Step S2.3.9: Loss of the output value of the online graph value network ogvn According to the gradient backpropagation method, the network parameter ω of the online graph value network is updated.

[0061] Step S2.3.10: Gradient value

number

number

number

[0062] Step S2.3.11: Update the network parameters θ′ of the target graph strategy network and ω′ of the target graph value network according to the following formulas: θ′=τθ+(1-τ)θ′ ω′=τω+(1-τ)ω′ where τ is a constant and τ∈(0, 1).

[0063] Step S2.3.12: Repeat steps S2.3.2 to S2.3.11 until the number of iterations reaches a preset number T, and obtain a routing policy that minimizes the routing overhead of the target SDN network.

[0064] Step S3: Using the trained deep graph learning model, obtain a routing policy that minimizes the routing overhead of the target SDN network based on the state of the target SDN network, deploy the routing policy to the target SDN network, and change the weight of each link in the target SDN network according to the routing policy, thereby completing the routing optimization of the target SDN network.

[0065] The specific steps of step S3 are as follows.

[0066] Step S31: Obtain a graph adjacency matrix A and a network information feature matrix H of the target SDN network; Step S32: Based on the trained deep graph learning model, obtain a routing policy that minimizes the routing overhead of the target SDN network according to the state [A, H] of the target SDN network; Step S33: Deploy the routing policy obtained in step S32 to the target SDN network, and change the weight of each link in the target SDN network according to the routing policy; Step S34: In the traffic transmission process, perform traffic transmission using the updated weight of each link according to the shortest path solution.

[0067] An embodiment of the present invention also provides a system for a routing optimization method based on graph structure characteristics. As shown in Figure 1, a target SDN network includes a control plane and a data plane, and the control plane includes an information acquisition module, a strategy deployment module, and a DGL module, so that the system for a routing optimization method based on graph structure characteristics realizes the routing optimization method based on graph structure characteristics.

[0068] Each link and each node of the target SDN network is arranged in the data plane, and the information acquisition module of the control plane is used to acquire the network topology graph of the target SDN network, generate a graph adjacency matrix and a network information feature matrix, and send them to the DGL module.

[0069] The DGL module is based on a graph learning neural network, and takes the graph adjacency matrix and the network information feature matrix as input, and uses the deep graph learning method to output the routing overhead of the target SDN network in the current state. The DGL module updates the network parameters of the graph learning neural network based on the gradient backpropagation method, and trains the graph learning neural network after a preset number of iterations to obtain a deep graph learning model that minimizes the routing overhead and maximizes the link utilization of the target SDN network.

[0070] The strategy placement module of the control plane is used to obtain a routing policy that minimizes the routing overhead of the target SDN network based on the state of the target SDN network using the trained deep graph learning model obtained by the DGL module, and send the routing policy and the routing overhead of the target SDN network to the data plane.

[0071] Although the embodiments of the present invention have been described in detail above with reference to the drawings, the present invention is not limited to the above embodiments, and various modifications are possible within the scope of knowledge possessed by those skilled in the art without departing from the spirit of the present invention.

Claims

1. A routing optimization method based on graph structure features, comprising: For the target SDN network, perform the following steps S1 to S3 to obtain the routing overhead of each link in the target SDN network and adjust the weight of each link to complete the routing optimization of the target SDN network; Step S1: For a target SDN network, obtain a network topology graph of the target SDN network based on the southbound interface protocol; construct a graph adjacency matrix according to the connection relationship between each node on each link of the target SDN network in the network topology graph; for each node on each link of the target SDN network, construct an information feature vector of each node based on the link bandwidth, traffic, packet loss rate, and transmission delay of each node; and construct a network information feature matrix of the target SDN network based on the information feature vector of each node; Step S2: Using the graph adjacency matrix and the network information feature matrix as the state of the target SDN network, based on the graph learning neural network, using the graph adjacency matrix and the network information feature matrix as inputs, and using the deep graph learning method to output the routing policy and routing overhead of the target SDN network in the current state, update the network parameters of the graph learning neural network based on the gradient backpropagation method, and train the graph learning neural network after a preset number of iterations to obtain a deep graph learning model that minimizes the routing overhead and maximizes the link utilization of the target SDN network; Step S3: Using the trained deep graph learning model, obtain a routing policy that minimizes the routing overhead of the target SDN network based on the state of the target SDN network, deploy the routing policy in the target SDN network, and change the weight of each link in the target SDN network according to the routing policy, thereby completing the routing optimization of the target SDN network.

2. The specific steps of step S1 are as follows: Step S1.1: For a target SDN network, obtain a network topology structure of the target SDN network according to a southbound interface protocol, where the network topology structure includes M routers and N links; Step S1.2: In the network topology structure of the target SDN network, each router corresponds to one real node, each link corresponds to one edge, and virtual nodes are inserted into the edges corresponding to each link. The network topology structure of the target SDN network is represented by a network topology graph G(V, E) having M real nodes, N virtual nodes, and 2N edges, where V represents a node set and E represents an edge set, and is specifically as shown in the following formula: V={V 実 、V 仮想 } Here, V 実 represents the real node set, and V 仮想 represents the set of virtual nodes, V 実 ={v ｓ１ 、v ｓ２ 、...、v ｓＭ } Here, v ｓ１ , v ｓ２ ,... ,v ｓＭ represents M real nodes, V 仮想 ={v ｘ１ 、v ｘ２ 、...、v ｘＮ } Here, v ｘ１ , v ｘ２ ,... ,v ｘＮ represents N virtual nodes, E={e １ 、e ２ 、...、e ２Ｎ } Here, e １ , e ２ ,... ,e ２Ｎ represents 2N edges, Step S1.3: Construct an x-th order graph adjacency matrix A based on the network topology graph of the target SDN network, where x=M+N, where x represents the total number of nodes and the nodes include M real nodes and N virtual nodes, as shown in the following formula: [Number 54] Here, element a in the graph adjacency matrix A ｉｊ is as follows: [Number 55] Step S1.4: For any node i in the target SDN network, based on the link bandwidth, traffic, packet loss rate, and transmission delay of node i, calculate the information feature vector h of node i, which is given by the following formula: ｉ Build h ｉ =[B ｗｉ 、T ｈｉ 、L ｐｉ 、D ｔｉ ] In the formula, B ｗｉ is the link bandwidth of node i, and T ｈｉ is the traffic of node i, and L ｐｉ is the packet loss rate of node i, and D ｔｉ is the transmission delay of node i, According to the information feature vector of each node, construct a network information feature matrix H of the target SDN network, as shown in the following formula: [Number 56] In the formula, h １ , h ２ ,... ,h ｉ ,... ,h ｘ 2. The routing optimization method based on graph structure features according to claim 1, wherein: is the information feature vector of each node.

3. For the node i in step S1.4, if node i is a virtual node, the traffic T ｈｉ , packet loss rate L ｐｉ , transmission delay D ｔｉ is 0, and if node i is a real node, the link bandwidth B ｗｉ 3. The routing optimization method based on graph structure characteristics according to claim 2, wherein: is 0.

4. The deep graph learning method in step S2 includes four graph learning neural networks and one experience pool, where the four graph learning neural networks are an online graph strategy network, an online graph value network, a target graph strategy network, and a target graph value network, respectively, and the four graph learning neural networks include one input layer, two hidden layers, and one output layer; The input layers of the online graph strategy network and the target graph strategy network take the graph adjacency matrix A and the network information feature matrix H as inputs, and the outputs of the online graph strategy network and the target graph strategy network are inputs to the online graph value network and the target graph value network, respectively. Here, the propagation formulas from the input layer to the hidden layer and between the hidden layers of each graph learning neural network are the same. If the input layer is the 0th layer, the first hidden layer is the 1st layer, and the second hidden layer is the 2nd layer, the propagation formula is as follows: [Number 57] where σ(·) indicates that the expression in the brackets is normalized, and H ｌ is the network information feature matrix of the lth layer, and W ｌ＋１ is the weight matrix of the l+1th layer, where H ０ = H, [Number 58] I is the x-th order identity matrix, [Number 59] where: [Number 60] Here, in the online graph strategy network and the target graph strategy network, W １ is a 4x4 matrix, and W ２ is a 4×1 matrix, the output layer is a fully connected layer, and its output value is an x×1 matrix. The routing policy is expressed as Policy, and is specifically expressed as follows: Policy=H ２ ×K where K is the weight matrix of the output layer of the online graph strategy network and the target graph strategy network, and H ２ is the network information feature matrix of the second layer, In the online graph value network and the target graph value network, １ and W ２ are all 1x1 matrices, the output layer is an aggregation layer, and its output value is a 1x1 matrix, denoted as Value, and specifically, is as shown in the following formula: [Number 61] where Q is the weight value of the output layer, [Number 62] is the second layer network information feature matrix H ２ is the i-th value in The routing optimization method based on graph structure characteristics according to claim 2, characterized in that the routing overhead of each link in the target SDN network is updated according to the routing policy Policy output by the online graph strategy network.

5. The specific steps of step S2 are as follows: Step S2.1: Initialize the weight matrices of the online graph strategy network, the online graph value network, the target strategy network, and the target graph value network, where the weight matrix of the online graph strategy network is W θ and the weight matrix of the online graph value network is W θ′ and the weight matrix of the target graph strategy network is W ω and the weight matrix of the target graph value network is W ω′ and Step S2.2: Initialize the experience pool, the specific steps are as follows: Step S2.2.1: Let the graph adjacency matrix A and the network information feature matrix H be the state S of the target SDN network, and define S = [A, H]. ｔ represents the state of the target SDN network at time t, and s ｔ = [A ｔ , H ｔ ] and A ｔ is the graph adjacency matrix of the target SDN network at time t, and H ｔ represents the network information feature matrix of the target SDN network at time t; Step S2.2.2: [Number 63] are defined as the outputs of the online graph strategy network, the target graph strategy network, the online graph value network, and the output layer of the target graph value network at time t, respectively, and the online graph strategy network routing policy is calculated by the following equation: [Number 64] The environmental feedback f ｔ Calculate f ｔ =U(B ｗ 、T ｈ 、L ｐ 、D ｔ )×K ｆ In the formula, U(B ｗ , T ｈ , L ｐ , D ｔ ) is the link utilization rate, and B ｗ , T ｈ , L ｐ , D ｔ are the link bandwidth, traffic, packet loss rate, and transmission delay of the target SDN network, respectively, and K ｆ is the proportionality coefficient, The target function U that maximizes the utilization of the target SDN network link ｍａｘ (B ｗ , T ｈ , L ｐ , D ｔ ) and Step S2.2.3: Define the experience pool R as follows: [Number 65] In the formula, s ｔ＋１ is the state of the target SDN network at time t+1, i.e., the online graph strategy network is [Number 66] represents the state of the target SDN network obtained by outputting Step S2.3: Iterate a predetermined number of times to the target SDN network, where the predetermined number of iterations is T, and the specific steps are as follows: Step S2.3.1: Let t=1, and then set the initial state s of the target SDN network. １ Get Step S2.3.2: The online graph strategy network determines the state s of the target SDN network at time t. ｔ Depending on the routing policy [Number 67] and execute the process. [Number 68] where θ is the network parameter of the online graph strategy network, Step S2.3.3: Routing policy [Number 69] Update the routing overhead of each link in the target SDN network according to Step S2.3.4: Routing policy [Number 70] The state of the target SDN network s is updated according to ｔ＋１ and environmental feedback f ｔ Get Step S2.3.5: [Number 71] as a set of historical records in an experience pool R, Step S2.3.6: Y sets of history records from experience pool R [Number 72] where the subscript m represents any one set of historical records in the experience pool R, Step S2.3.7: History record retrieved in step S2.3.6 [Number 73] Based on this, the output corresponding to the target graph value network is calculated as follows: [Number 74] Calculate [Number 75] is the state s of the target SDN network when the target graph strategy network is ｍ＋１ where θ′ is the network parameter of the target graph strategy network, ω′ is the network parameter of the target graph value network, and [Number 76] is the state of the target graph value network in the target SDN network. ｍ＋１ , and the network parameters are ω′, the routing policy π′(s ｍ＋１ |θ′), γ is the discount rate, a constant, and γ∈(0, 1), Step S2.3.8: The loss of the output value of the online graph value network is calculated by the following formula: ｏｇｖｎ Calculate [Number 77] is the state s of the target SDN network when the online graph value network with network parameters ω ｍ , the routing policy output by the online graph strategy network is π(s ｍ |θ), and Step S2.3.9: Loss of output value of online graph value network ｏｇｖｎ According to the gradient backpropagation method, update the network parameter ω of the online graph value network; Step S2.3.10: Gradient value [Number 78] Calculate the gradient value [Number 79] According to the gradient backpropagation method, the network parameters θ of the online graph strategy network are updated, where [Number 80] means to find the gradient of the expression in parentheses, Step S2.3.11: Update the network parameters θ′ of the target graph strategy network and ω′ of the target graph value network by the following formulas, respectively: θ'=τθ+(1-τ)θ' ω′=τω+(1−τ)ω′ where τ is a constant and τ∈(0,1); Step S2.3.12: Repeat steps S2.3.2 to S2.3.11 until the number of iterations reaches a preset number T, thereby obtaining a routing policy that minimizes the routing overhead of the target SDN network.

6. The specific steps of step S3 are as follows: Step S31: Obtain the graph adjacency matrix A and the network information feature matrix H of the target SDN network; Step S32: Based on the trained deep graph learning model, obtain a routing policy that minimizes the routing overhead of the target SDN network according to the state [A, H] of the target SDN network; Step S33: Deploy the routing policy obtained in step S32 to the target SDN network, and change the weight of each link in the target SDN network according to the routing policy; Step S34: In the traffic transmission process, the updated weight of each link is used to transmit traffic according to the shortest path solution.

7. A system for a routing optimization method based on graph structure features, comprising: The target SDN network includes a control plane and a data plane, and the control plane includes an information acquisition module, a strategy placement module, and a DGL module, so that the system for routing optimization method based on graph structure characteristics realizes the routing optimization method based on graph structure characteristics according to any one of claims 1 to 6; Each link and each node of the target SDN network is arranged in the data plane, and the information acquisition module of the control plane is used to acquire the network topology graph of the target SDN network, generate a graph adjacency matrix and a network information feature matrix, and send them to the DGL module; The DGL module uses the graph learning neural network as input, the graph adjacency matrix and the network information feature matrix, and uses the deep graph learning method to update the network parameters of the graph learning neural network using the gradient backpropagation method. After a predetermined number of iterations, the graph learning neural network is trained to obtain a deep graph learning model that minimizes the routing overhead and maximizes the link utilization of the target SDN network. A system for a routing optimization method based on graph structure characteristics, characterized in that the strategy placement module of the control plane is used to obtain a routing policy that minimizes the routing overhead of the target SDN network based on the state of the target SDN network using the trained deep graph learning model obtained in the DGL module, and to transmit the routing policy and the routing overhead of the target SDN network to the data plane.

Images