A SDN routing method based on a gated recurrent unit and a graph neural network

Abstract

The application provides an SDN routing method based on a gated recurrent unit and a graph neural network, and relates to the technical field of computer network routing. The graph neural network is introduced to model the network; the attention mechanism is introduced to realize different weight aggregation of different neighbor information; the GRU and the attention mechanism are combined to propose a gated recurrent unit model based on the attention mechanism, and the gated recurrent unit model based on the attention mechanism is used to replace the traditional RNN recurrent unit in the graph neural network, so that the gradient disappearance problem of the RNN is solved, intelligent routing decision for the software defined network is realized, and the performance of the network is improved under the condition of large-scale network and increased traffic.

Description

Technical Field

[0001] This invention relates to the field of computer network routing technology, and in particular to an SDN routing method based on gated recurrent units and graph neural networks. Background Technology

[0002] SDN topology modeling based on graph neural networks has become a research hotspot due to its conformity to the topology of computer networks. Lin employs a distributed hierarchical control plane architecture, designing a three-tier design consisting of a super controller, a master controller, and slave controllers. Simultaneously, combining reinforcement learning algorithms, a QoS-aware adaptive routing algorithm is designed, achieving efficient, adaptive, and QoS-preset packet forwarding. Younus applies reinforcement learning to the SDN controller to generate routing tables, intelligently learn routing paths, and improve network convergence speed and performance. Al-Jawad implements a trade-off between QoS and Quality of Experience (QoE) in the Q-Learning algorithm.

[0003] However, when aggregating information about different neighboring edges of the same side, an equivalent aggregation method is often used, that is, applying the same weight to all neighboring edges. This limits the model's ability to capture spatially relevant information. To overcome this problem, the Attention Mechanism (AM) has been introduced into deep learning models. The Attention Mechanism allows the model to selectively focus on the parts of the input data most relevant to a given task. It enables the model to assign different weights to different parts of the input, thereby learning the input parts most important for prediction. This is particularly crucial when dealing with large-scale and complex input tasks. Furthermore, the Attention Mechanism has efficient parallel processing capabilities. This mechanism is directly applicable to inductive learning problems and can even be generalized to tasks involving completely unknown graphs, providing a foundation for research involving tasks such as dynamically adding computer network nodes.

[0004] The most common attention mechanism is soft attention, which involves calculating a set of weights for the input data at each step of the computation. These weights are used to compute a weighted sum of the input data and serve as input for the next computation. This allows the model to selectively process different parts of the input data, weighting them according to their relevance to the current task. Another type of attention mechanism is hard attention, which selects a single part of the input data at each step of the computation, rather than computing a weighted sum. This is very useful for tasks dealing with structured input data, as it easily identifies the most relevant parts at each step. In general, attention mechanisms have proven to be powerful tools in deep learning, particularly in natural language processing tasks such as machine translation, sentiment analysis, and question answering, significantly improving accuracy by allowing the model to selectively focus on different parts of the input data. Therefore, attention mechanisms can be introduced to address the problem of different neighbor information being weighted differently when aggregating neighbor feature information in link aggregation.

[0005] Traditional routing methods typically use Recurrent Neural Networks (RNNs) as the neural network for modeling. An RNN has a storage unit, often called a hidden state or cell state, used to store previous input information. Through a feedback loop, the output of the RNN's hidden state can be fed back into the network as input for the next time step. This feedback loop allows the RNN to handle sequential data and make predictions based on the context of previous inputs. However, as time progresses, earlier memory data, after multiple rounds of processing with the tanh function, can lead to vanishing gradients or exponentially decreasing gradients, thus hindering the model from learning long-term dependencies. To mitigate the vanishing gradient problem, one approach is to use an activation function with a larger gradient than the tanh function, such as ReLU or its variants. However, if the weights are not initialized correctly or the learning rate is too high, it can lead to exploding gradients, causing the model to diverge during training. Summary of the Invention

[0006] The technical problem this invention aims to solve is to address the shortcomings of existing technologies by providing an SDN routing method based on gated recurrent units (GRUs) and graph neural networks. It introduces graph neural networks to model the network; introduces an attention mechanism to aggregate different weights of information from different neighbors; combines GRUs and the attention mechanism to propose an attention-based gated recurrent unit model, which replaces the traditional RNN recurrent unit in the graph neural network, solving the gradient vanishing problem of RNNs and achieving intelligent routing decisions for software-defined networks.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0008] An SDN routing method based on gated recurrent units and graph neural networks first extracts features on the software-defined network to achieve network topology modeling; then, the feature vectors of the modeled network topology links are applied to a dual deep Q network intelligent routing decision algorithm to perform intelligent routing decisions for SDN.

[0009] Among them, the attention mechanism is used to solve the problem of equally aggregating neighbor feature information on links. The attention mechanism is introduced into the aggregation of neighbor feature information on links to achieve aggregation of different neighbor information with different weights.

[0010] Furthermore, the attention mechanism focuses on the degree of association between each link and its neighbors, and uses attention scores to determine the amount of feature information aggregated by each neighbor link in the current link; firstly, for link l at time t... i After performing a linear transformation on the neighbor link feature information, its relationship to link l is calculated. i Contribution Secondly, for link l i The feature information of all neighbor links is normalized to obtain the weights. Finally, link l is weighted according to its weight. i A new feature is obtained by weighting the features of the neighbors.

[0011] Furthermore, the detailed process and formula of the attention mechanism are as follows:

[0012] For the input link l i Features of the tth cycle Features of t cycles of links with its neighbors After linear transformation, as shown in equations (1) and (2);

[0013]

[0014]

[0015] Among them, W t For attention weights, For link l i The t-th iteration contains the feature vectors with attention weights. The feature vector containing attention weights for the t-th iteration of its neighbor links;

[0016] Therefore, the characteristics of the neighbor link in the tth cycle are relevant to link l. i Contribution As shown in equation (3);

[0017]

[0018] Where LeakyReLU is the activation function, and α is the activation function parameter;

[0019] Use activation units to perform nonlinear transformation; link l i The correlation between the link and its neighboring links calculated in the tth iteration is normalized in a unified manner to better distribute the weights among different links, as shown in Equation (4).

[0020]

[0021] in, for The feature weights of the corresponding link obtained by the softmax function in the tth iteration; for link l i The neighbor features are weighted and averaged, and then subjected to a nonlinear transformation to obtain new features, as shown in Equation (5).

[0022]

[0023] Furthermore, after introducing the attention mechanism into the link aggregation neighbor feature information, the attention mechanism is combined with GRU to propose an attention mechanism-based gated recurrent unit model; the attention mechanism-based gated recurrent unit model is used to replace the RNN in the graph neural network, so that the attention mechanism is regarded as one of the gates of the gated graph recurrent neural network; therefore, the gated recurrent unit model consists of 3 gates: attention gate, reset gate, and update gate.

[0024] Furthermore, in the design of the gated recurrent unit, the RNN is expanded in a fixed time step T, and the gradient is calculated using a time-series backpropagation algorithm; link l i GRU aggregates information from its neighboring links, using information from each neighbor and the previous time step to update the hidden state. z and r are the update gate and reset gate, respectively. It is Hardamard bitwise multiplication;

[0025] The specific calculation formulas are shown in equations (6) to (10);

[0026]

[0027] in, For link l i The feature information at time t-1, A t-1 For link l i The feature information of the neighbor link matrix at time t-1, W a The weights of the attention gates;

[0028]

[0029] Among them, W r To reset the gate weights; a t Input for the current time step; U r To reset the gate weight matrix, used to adjust the feature information of the previous time step. Impact on the reset door; b r The bias vector for the reset gate; the reset gate r is computed using the sigmoid activation function. t ;

[0030]

[0031] Among them, W z To update the gate weights; U z To update the gate weight matrix, which is used to adjust the feature information from the previous time step. Impact on the update gate; b z To update the bias vector of the gate, use the sigmoid activation function to update the gate z. t ;

[0032]

[0033] Among them, W h Link feature information The weight matrix is ​​used to adjust the input a at the current time step. t Impact on candidate hidden states; U h This is the weight matrix, used to adjust the reset gate r. t Feature information from the previous time step The effect on candidate hidden states; ⊙ represents the dot product operation; b h The bias term is used to adjust the offset of the candidate hidden state; the candidate hidden state is obtained after passing through the tanh function.

[0034]

[0035] Among them, 1-z t In a state of forgetting, In order to selectively forget existing information, This indicates selective memorization of candidate hidden state information.

[0036] Furthermore, after replacing the RNN model in the graph neural network with the attention-based gated recurrent unit model, at time t, link l i The result of the interaction between the feature information and the feature information of its neighbor link matrix is ​​used as the input to the Double Deep Q-Network (DDQN) intelligent routing decision algorithm.

[0037] Furthermore, the DDQN intelligent routing decision algorithm selects a suitable path from the action set based on the Q-value according to the input network performance state, and applies it to the environment to obtain a new network performance state. The network performance state, path, and reward value are stored together in the experience pool to obtain samples for training, and the trained DDQN model is stored in the knowledge base. Finally, the model outputs the intelligent routing decision path, and uses network performance latency as the reward value to train the DDQN intelligent routing decision algorithm.

[0038] The beneficial effects of adopting the above technical solution are as follows: The SDN routing method based on gated recurrent units and graph neural networks provided by this invention includes two optimizations to existing graph neural network-based routing methods. First, it optimizes the link aggregation of neighbor information using an attention mechanism. Second, it replaces the RNN in the graph recurrent neural network with a GRU. Network modeling is performed using a gated graph recurrent neural network based on an attention mechanism, and then the graph data structure information of the output network is input into the DDQN routing algorithm for intelligent routing decision-making, with the output being the routing path. This achieves intelligent routing decision-making for software-defined networks, improving network performance under large-scale network and traffic increases. Attached Figure Description

[0039] Figure 1 The main flowchart of the attention-based gated loop unit model provided in the embodiments of the present invention is shown below.

[0040] Figure 2 A diagram illustrating the attention mechanism model provided in an embodiment of the present invention;

[0041] Figure 3 This is a model diagram of a gated loop unit provided in an embodiment of the present invention. Detailed Implementation

[0042] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0043] like Figure 1 As shown, the method of this embodiment is described below.

[0044] This method primarily builds upon the gated recurrent unit (RNU) model by integrating the attention mechanism as an attention gate into the RNU model. It proposes an attention-based gated recurrent unit model and uses it to replace the traditional RNN recurrent unit in graph neural networks, thus solving the gradient vanishing problem in RNNs and achieving intelligent routing decisions for software-defined networks. Therefore, the routing method using the gated recurrent unit and graph neural network mainly includes three gates: the attention gate, the reset gate, and the update gate.

[0045] The overall process involves feature extraction on the software-defined network (SDN) using a routing method based on gated recurrent units and graph neural networks, followed by network topology modeling. The feature vectors of the modeled network topology links are then applied to the DDQN intelligent routing decision algorithm to perform intelligent routing decisions for SDN.

[0046] First, to address the issue of equally aggregating neighbor feature information across links, an attention mechanism is used. Second, to solve the gradient vanishing problem in RNNs, a gated recurrent unit model based on an attention mechanism is used instead. These two parts will be described below.

[0047] To address the problem of equivalent aggregation of neighbor information in graph neural networks, an attention mechanism is introduced to achieve aggregation using different weights for different neighbor information. The attention mechanism model in this embodiment is as follows: Figure 2 As shown. The goal of the attention mechanism is to learn the relevant information between links, retaining information about important neighbor links and reducing information about unimportant neighbor links. Specifically, it mainly focuses on the degree of correlation between each link and its neighbors, and uses attention scores to determine the amount of feature information aggregated by each neighbor link in the current link. First, for link l at time t... i After performing a linear transformation on the neighbor link feature information, its relationship to link l is calculated. i Contribution Secondly, for link l i The feature information of all neighbor links is normalized to obtain the weights. Finally, link l is weighted according to its weight. i A new feature is obtained by weighting the features of the neighbors. The detailed process and formula of this method are as follows.

[0048] For the input link l i Features of the tth cycle Features of t cycles of links with its neighbors After linear transformation, as shown in equations (1) and (2).

[0049]

[0050]

[0051] Among them, W t For attention weights, For link l i The t-th iteration contains the feature vectors with attention weights. The feature vector containing attention weights for the t-th iteration of its neighbor links.

[0052] Therefore, the characteristics of the neighbor link in the tth cycle are relevant to link l.i The contribution is shown in equation (3).

[0053]

[0054] Where LeakyReLU is the activation function, and α is the activation function parameter, which is typically 0.01.

[0055] Nonlinear transformation is performed using activation units. Link l i The correlation between the link and its neighboring links calculated in the tth iteration is normalized in a unified manner to better distribute the weights among different links, as shown in Equation (4).

[0056]

[0057] Among them, the feature weights of the corresponding link are obtained after t iterations. For link l i The neighbor features are weighted and averaged, and then subjected to a nonlinear transformation to obtain new features, as shown in Equation (5).

[0058]

[0059] After introducing an attention mechanism to aggregate neighbor feature information in the link, a gated recurrent unit (GRU) model based on the attention mechanism is used to replace the RNN in the graph neural network, thus treating the attention mechanism as one of the gates in the gated graph recurrent neural network. Therefore, the gated recurrent unit and the graph neural network mainly consist of three gates: the attention gate, the reset gate, and the update gate. After replacement with GRU, at time t, link l... i The result of the interaction between the feature information and the feature information of its neighbor link matrix is ​​used as input.

[0060] The model of the gated loop unit in this embodiment is as follows: Figure 3 As shown. An RNN is expanded over a fixed time step T, and the gradient is calculated using a temporal backpropagation algorithm; link l i GRU aggregates information from its neighboring links, using information from each neighbor and the previous time step to update the hidden state. z and r are the update gate and reset gate, respectively. It is the Hardamard bitwise multiplication operation. The specific calculation formulas are shown in equations (6) to (10).

[0061]

[0062] in, For link l i The feature information at time t-1, A t-1 For link l iThe feature information of the neighbor link matrix at time t-1, W a The weight of the attention gate.

[0063]

[0064] Among them, W r To reset the gate weights; a t Input for the current time step; U r To reset the gate weight matrix, used to adjust the feature information of the previous time step. Impact on the reset door; b r The bias vector for the reset gate; the reset gate r is computed using the sigmoid activation function. t .

[0065]

[0066] Among them, W z To update the gate weights; U z To update the gate weight matrix, which is used to adjust the feature information from the previous time step. Impact on the update gate; b z To update the bias vector of the gate, use the sigmoid activation function to update the gate z. t .

[0067]

[0068] Among them, W h Link feature information The weight matrix is ​​used to adjust the input a at the current time step. t Impact on candidate hidden states; U h This is the weight matrix, used to adjust the reset gate r. t Feature information from the previous time step The effect on candidate hidden states; ⊙ represents the dot product operation; b h The bias term is used to adjust the offset of the candidate hidden state; the candidate hidden state is obtained after passing through the tanh function.

[0069]

[0070] Among them, 1-z t In a state of forgetting, In order to selectively forget existing information, This indicates selective memorization of candidate hidden state information.

[0071] After replacing the RNN model in the graph neural network with a gated recurrent unit model based on an attention mechanism, the link l will be... iThe result of the interaction between the feature information and the feature information of its neighbor link matrix is ​​used as the input to the Double Deep Q-Network (DDQN) intelligent routing decision algorithm.

[0072] The DDQN routing decision algorithm selects an appropriate path from the action set based on the Q-value according to the input network performance state and applies it to the environment to obtain a new network performance state. The network performance state, path, and reward value are stored together in an experience pool to obtain samples for training, and the trained DDQN model is stored in a knowledge base. Finally, the model outputs the intelligent routing decision path. Network performance latency is used as the reward value to train the DDQN intelligent routing decision algorithm.

[0073] The experimental simulation system consists of a Mininet and a Ryu controller. In the test network, each host acts as a traffic initiator, generating traffic destined for other hosts. After the Mininet environment of the SDN network is started, each host randomly sends data packets to any other host in the network with equal probability. All experiments were run on a machine with an Intel(R) Core(TM) i5-10210U CPU @ 1.60GHz 2.11GHz, using Ubuntu 18.04 as the operating system. In the experimental simulation system, the graph neural network and reinforcement learning frameworks were implemented using Tensorflow in Python. The network topology of the experiment was a custom SDN network topology based on 3 Ryu controllers, 12 nodes, and 18 links. Various routing algorithms were deployed on the knowledge plane of the controllers to evaluate the performance of each algorithm.

[0074] Compared with existing technologies, the method proposed in this invention exhibits good stability on SDN networks, with lower network latency jitter than existing graph neural network-based routing methods. The network packet loss rate is also lower. Experiments confirm that, regardless of whether the traffic is 7.5M or 10M, the routing method based on gated recurrent units (GRUs) and graph neural networks outperforms existing graph neural network-based routing methods in terms of packet loss rate performance. As the network topology scale increases, the average latency of both the GRU-based and graph neural network-based routing algorithms and existing graph neural network-based routing methods gradually increases. However, at all network scales, the average latency of the GRU-based routing algorithm is consistently lower than that of existing graph neural network-based routing methods. Moreover, as the network scale increases, the gap between the average latency generated by the GRU-based routing algorithm and existing graph neural network-based routing methods widens. Experiments demonstrate that when the network scale increases, the GRU-based routing method exhibits good network performance, with its average latency improved by 4.2%-9.8% compared to existing graph neural network-based routing methods.

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. An SDN routing method based on gated recurrent units and graph neural networks, characterized in that: The method first extracts features on the software-defined network to achieve network topology modeling; then, it applies the feature vectors of the modeled network topology links to the dual deep Q network intelligent routing decision algorithm to perform intelligent routing decisions for SDN. Among them, the attention mechanism is used to solve the problem of equally aggregating neighbor feature information in the link, so as to achieve aggregation of different neighbor information with different weights; The attention mechanism focuses on the degree of association between each link and its neighbors, and uses attention scores to determine the amount of feature information aggregated by each neighbor link in the current link; firstly, for the links at time t... After performing a linear transformation on the neighbor link feature information, its relationship to the link is calculated. Contribution Secondly, regarding the link The feature information of all neighbor links is normalized to obtain the weights. Finally, the links are weighted. A new feature is obtained by weighting the features of the neighbors. ; After introducing the attention mechanism into the link aggregation neighbor feature information, the attention mechanism is combined with GRU to propose an attention-based gated recurrent unit model. The attention-based gated recurrent unit model is used to replace the RNN in the graph neural network, so that the attention mechanism is regarded as one of the gates of the gated graph recurrent neural network. Therefore, the gated recurrent unit model consists of 3 gates: attention gate, reset gate, and update gate.

2. The SDN routing method based on gated recurrent units and graph neural networks according to claim 1, characterized in that: The detailed process and formula of the attention mechanism are as follows: For the input link Features of the tth cycle The characteristics of t cycles of the link with its neighbors. After linear transformation, as shown in equations (1) and (2); (1)； (2)； in, For attention weights, For link The t-th iteration contains the feature vectors with attention weights. The feature vector containing attention weights for the t-th iteration of its neighbor links; Therefore, the characteristics of the neighbor link in the tth cycle are relevant to the link. Contribution As shown in equation (3); (3)； in, For activation function, These are the parameters for the activation function; Use activation units to perform nonlinear transformations; link The correlation between the link and its neighboring links calculated in the tth iteration is normalized in a unified manner to better distribute the weights among different links, as shown in Equation (4). (4)； in, The feature weights for the corresponding link obtained through the softmax function in the tth iteration; for the link The neighbor features are weighted and averaged, and then subjected to a nonlinear transformation to obtain new features, as shown in Equation (5). (5)。 3. The SDN routing method based on gated recurrent units and graph neural networks according to claim 1, characterized in that: In the design of the gated recurrent unit, the RNN is expanded in a fixed time step T, and the gradient is calculated using a time-series backpropagation algorithm; link GRU uses information from each neighbor and the previous time step to update the hidden state by aggregating information from its neighboring links. z and r are the update gate and reset gate, respectively, and ⨁ is the Hardamard bitwise product operation. The specific calculation formulas are shown in equations (6) to (10); (6)； in, For link The feature information at time t−1, For link The feature information of the neighbor link matrix at time t−1, The weights of the attention gates; (7)； in, To reset the gate weight; Input for the current time step; To reset the gate weight matrix, used to adjust the feature information of the previous time step. Impact on the reset door; To reset the gate's bias vector; use the sigmoid activation function to compute the reset gate. ; (8)； in, To update the gate weights; To update the gate weight matrix, which is used to adjust the feature information from the previous time step. Impact on the door update; To update the gate's bias vector, use the sigmoid activation function to update the gate. ; (9)； in, Link feature information The weight matrix is ​​used to adjust the input at the current time step. The impact on candidate hidden states; This is a weight matrix used to adjust the reset gate. Feature information from the previous time step The impact on candidate hidden states; This represents the dot product operation; The bias term is used to adjust the offset of the candidate hidden state; the candidate hidden state is obtained through the tanh function. ; (10)； in, In a state of forgetting, In order to selectively forget existing information, This indicates selective memorization of candidate hidden state information.

4. The SDN routing method based on gated recurrent units and graph neural networks according to claim 1, characterized in that: After replacing the RNN model in the graph neural network with the attention-based gated recurrent unit model, the link will be... The result of the interaction between the feature information and the feature information of its neighbor link matrix is ​​used as the input to the dual-depth Q network intelligent routing decision algorithm.

5. The SDN routing method based on gated recurrent units and graph neural networks according to claim 4, characterized in that: The Dual Deep Q-Network Intelligent Routing Decision Algorithm selects an appropriate path from the action set based on the Q-value according to the input network performance state and applies it to the environment to obtain a new network performance state. The network performance state, path, and reward value are stored together in the experience pool to obtain samples for training, and the trained DDQN model is stored in the knowledge base. Finally, the model outputs the intelligent routing decision path and uses network performance latency as the reward value to train the DDQN intelligent routing decision algorithm.

Images