SDN-architecture-based routing method for guaranteeing network QOS

Abstract

Disclosed in the present invention is an SDN-architecture-based routing method for guaranteeing network QoS. The method comprises: on the basis of network topology information, performing abstraction to obtain a directed graph; then, on the basis of an improved K-means algorithm, dividing data into mouse flows, elephant flows and mammoth flows; defining a traffic transmission constraint condition; constructing a DQN neural network; training the DQN neural network by using the maximum Q value, which corresponds to an action a having the maximum reward function R, as an objective, so as to obtain a QoS guarantee routing model; obtaining an optimal path weight value; determining a data packet sending path on the basis a Dijkstra algorithm; and finally issuing path information to a switch. The present invention avoids the situations in which data processing performed by using a conventional K-means algorithm is not stable enough and a classification result is not accurate enough. During the training of a DQN neural network, data is extracted from an experience pool according to the priority of data, such that the neural network pays more attention to important data, thereby improving the learning efficiency of the neural network and the performance of the algorithm.

Description

A network QoS guarantee routing method based on SDN architecture Technical Field

[0001] The present invention belongs to the field of network technology, and in particular relates to a network QoS guarantee routing method based on SDN architecture. Background Art

[0002] As networks continue to expand, problems like congestion and latency are becoming increasingly prominent. Traditional networks, with their tightly coupled control and forwarding architectures and "best-effort" QoS service model, are unable to dynamically adjust and optimize network traffic, making it difficult to meet the QoS requirements of diverse application scenarios. Software-Defined Networking (SDN) technology, by separating the network control and data planes, enables dynamic adjustment of network traffic, improving network performance and QoS assurance.

[0003] Traffic in data center networks is generally categorized as "elephant flows" and "rat flows" based on data volume. However, the smaller number of "elephant flows" carries the majority of network traffic, while the larger number of "rat flows" carries only a small portion. Currently, many traffic classification approaches use static or dynamic thresholds to identify "elephant flows" and "rat flows," and use the same forwarding method for all "elephant flows." However, the size of "elephant flows" varies significantly, and forwarding very large "elephant flows" together with regular "elephant flows" can also cause localized network congestion and load imbalance.

[0004] The K-means algorithm is one of the most widely used machine learning algorithms for network traffic classification. However, the classic K-means algorithm also has some shortcomings. For example, the randomness of the initial cluster center selection makes the algorithm sensitive to abnormal data, which has a significant impact on classification accuracy. Due to the randomness of the initial cluster centers, the traditional K-means algorithm is not robust in processing data, which may lead to inaccurate classification results.

[0005] OSPF and BGP are two of the most commonly used QoS-based routing algorithms, but each has its own shortcomings. OSPF can only statically select a single path for data packets and cannot achieve load balancing. While BGP supports multiple paths, it requires configuring numerous policies and routing rules, makes routing decisions more slowly, and limits network scalability.

[0006] Summary of the Invention

[0007] The technical problem to be solved by the present invention is that: in the prior art, OSPF can only statically select a single path as the routing path for data packets and cannot achieve load balancing; although BGP supports multiple paths, it requires the configuration of more policies and routing rules, and the routing decision is slow, which also has certain limitations on the scalability of the network. A network QoS guarantee routing method based on SDN architecture is provided to better realize the flexible on-demand allocation of data center network resources, maximize the utilization of data center network resources, and improve network performance and QoS guarantee.

[0008] To solve the above technical problems, the present invention provides the following technical solution: a network QoS guaranteed routing method based on SDN architecture, comprising the following steps:

[0009] S1. The SDN controller periodically obtains data center network topology information according to the preset time t;

[0010] S2. Based on the topology information, the data center network is abstracted into a directed graph. The data transmission path of the data center network is obtained through the directed graph and represented by a weight matrix.

[0011] S3: The SDN controller receives the data stream from the source host and determines whether the destination node to which the data stream is to be sent is directly connected to the source node based on the directed graph. If so, the data packet is forwarded directly along the network reachable path; otherwise, step S4 is executed.

[0012] S4. Using an improved k-means algorithm based on data distribution density to cluster the data streams, that is, first using the traffic distribution density function to determine the initial cluster center, and then clustering the data streams into data streams containing various classification features, namely, mouse streams, elephant streams, and giant elephant streams;

[0013] S5. Calculate the end-to-end delay, available bandwidth, and packet loss rate of the path, and then construct the QoS constraint of the data flow, that is, the minimum total transmission cost Cost. min ;

[0014] S6. Construct a DQN neural network, including a Q network and a target Q_target network; define the state space, including network topology information, source nodes, destination nodes, and data streams of various classification features; define the action space A for the interaction between the agent and the environment as the weight value for the data stream to select the path transmission between nodes, and the reward function R as the negative correlation of the minimum total transmission cost of each path in the network. The DQN neural network is trained with the maximum Q value corresponding to the action a with the largest reward function R as the target. During the training process, the experience replay mechanism and the target Q_target network are used to optimize the routing strategy to obtain a QoS guaranteed routing model;

[0015] S7. Using the QoS guarantee routing model, obtain the weight value of the optimal path for data flow transmission, obtain the optimal path for corresponding data flow transmission based on the weight value, and further determine all nodes that the transmission path passes through;

[0016] S8. According to the Dijkstra algorithm, the node with the smallest weight value among all the nodes passed by the transmission path is selected as the routing node to determine the transmission path of the data stream.

[0017] Furthermore, in the aforementioned step S1, the SDN controller communicates with the switches in the network through the southbound OpenFlow protocol to obtain the connection relationship and link information of the switches, and periodically obtains and updates the global network topology information through the link discovery protocol LLDP. The SDN network includes x controllers and y switches, and the set of x controllers is C = {c1, c2, ..., c x}, and set the controller c1 as the central controller, and the set of y switches is S = {s1,s2,……,s y}.

[0018] Furthermore, in the aforementioned step S2, the directed graph is G = (U, V, W), where U represents the set of nodes u in the network, i.e., u∈U; V represents the set of links v in the network, and the connection between two nodes defines a link, i.e., v∈V. Represents the weight matrix of the network path node at the initial time, where w ij =R + , that is, a positive real number, indicating that there is a path connecting nodes i and j, that is, data is reachable; w ij =0, indicating that there is no path connecting i and j, i.e., the data is unreachable.

[0019] Furthermore, the aforementioned step S4 includes the following sub-steps:

[0020] S41, for any node i in the network, calculate the Euclidean distance dis(l a ,l b ), as follows:

[0021] The data flow transmitted by node i in the network is represented by the vector set L={l i |l i ∈R p ,i=1,2,3,…,n}, n is the number of traffic transmitted in node i, p is the representation dimension of the traffic, l a and l b are any two data flows in the traffic set L;

[0022] Calculate the average Euclidean distance of all data flows of node i As follows:

[0023] in, The smaller the value, the smaller the average Euclidean distance of the data streams, which means the difference between the data streams is smaller, that is, the data streams of the node are more similar;

[0024] S42, calculate the data flow l in the traffic set L transmitted by node i a The data density den(l a ), as follows:

[0025] Among them, den(l a ) indicates that in data stream l a The transmission time t(l a ) the amount of data or the number of data packets transmitted, den(l a ) is used to measure the traffic intensity of the data flow;

[0026] Calculate the average data density of all data flows at node i As follows:

[0027] in, The larger the value, the greater the average data density of the data flow, which means that the proportion of data transmitted in node i is higher, that is, the more concentrated the traffic distribution is;

[0028] The average transmission time of n data streams transmitted by computing node i is As follows:

[0029] Compute the cluster center c corresponding to all data flows of node i i , as follows:

[0030] Among them, c i represents the common characteristics of all data flows of node i in terms of size, distance, density, etc.

[0031] S43, calculate the traffic distribution function F(l i ), as follows:

[0032] Where P() is the probability function,

[0033] Calculate the distribution density function f(l i ), as follows:

[0034] Among them, f(l i ) represents the flow rate l i The probability of being distributed within a certain interval;

[0035] S44. Repeat steps S41-S43 for all U nodes in the network according to the directed graph G = (U, V, W), obtain the distribution density function of the transmission flow of each node in the network, and use the set f all It is represented as follows: all ={f1,f2,…,f i ,…f u},

[0036] The cluster mean center C corresponding to each node all As follows: C all ={c1,c2,…,c i ,…,c u},

[0037] Average traffic distribution density function of all U nodes in the network As follows:

[0038] Average flow distribution density function It is used to measure the average value of the traffic density of each node in the network. By comparing with the value of this function, each node in the network is divided into high-density nodes f i h and low-density nodes f i l ;

[0039] S45, according to the high-density node f i h and low-density nodes f i l , and further get the following formula:

[0040] The cluster center corresponding to the high-density node is moved from C all Extracted from the set C as the preliminary value of the K-means clustering center, and h Indicated as follows: C h ={c1,c2…,c i …,c m},i=1,2,3…m,m<n,

[0041] S46, from C h Find the data object c1 with the largest value as the first initial cluster center And change c1 from C h Delete in;

[0042] S47, then from C h Find the distance The farthest data object c2 is used as the second initial cluster center And change c2 from C h Delete in;

[0043] S48, from C h Find the distance and The farthest data object c3 is used as the third initial cluster center And change c3 from C h Delete in;

[0044] S49, assigning data streams to corresponding data clusters: using the initial cluster center set obtained in steps S46-S48 Cluster each data stream of each node in the network, calculate its distance from each initial cluster center, and assign it to the data cluster corresponding to the nearest cluster center;

[0045] S410, update cluster center: for each cluster, repeat steps S41-S42, calculate the average value of all data streams assigned to the cluster, that is, the center point of the data stream, and use the average value as the new cluster center

[0046] S411, repeat steps S44-S410 until the cluster center no longer changes, save the clustering results, and cluster the data flow into mouse flow L r 、Elephant Flow L e 、Elephant Flow L g .

[0047] Furthermore, in the aforementioned step S5, the QoS constraint conditions of the data flow are obtained according to the following steps:

[0048] S51. Calculate the end-to-end delay T of path p(i,j) d (p), as follows:

[0049] Among them, T t (v) T c (v) T q (v) are the transmission delay, propagation delay, and queuing delay of link v in path p, respectively;

[0050] S52. Calculate the available bandwidth B(p) of path p, as follows:

[0051] Where t is the time interval for the preset SDN controller to obtain the network topology information of the network data center; D r is the amount of data received by the port in the current time interval t; D′ t is the amount of data sent by the port in the previous time interval t, B cis the channel bandwidth defined based on Shannon's theory;

[0052] S53. Calculate the packet loss rate of path p:

[0053] Among them, D t-all is the total amount of data sent by the port in the current time interval t; D r is the amount of data received by the port in the current time interval t, and the difference between the two is the packet loss;

[0054] S54, calculate the minimum transmission cost of path p min (p), as follows: Cost min (p) = [αT d (p)-βB(p)+γLo(p)],

[0055] Among them, α, β, γ are the weight coefficients of link cost, and the minimum cost Cost min (p) Taking into account the combined effects of link bandwidth, delay and packet loss rate, the minimum cost is controlled min (p) to achieve the best performance of network transmission.

[0056] Furthermore, the aforementioned step S6 includes the following sub-steps:

[0057] S61, mapping the parameters of the DQN neural network to the actual application scenario of the SDN network, including the state space S, action space A and reward function R;

[0058] Among them, the state space S includes network topology information, source nodes, destination nodes, and data streams of various classification features; the action space A of the interaction between the agent and the environment is defined as follows:

[0059] a(s t ) is the weight value of the data flow l choosing the pth path between node i and node j for transmission There are K paths between node i and node j. Data stream l is transmitted on the pth path between node i and node j. The weight of the pth path is:

[0060] in, represents the traffic transmitted on path p between node i and node j;

[0061] The reward function R is the negative correlation of the minimum transmission cost of each path in the network, as shown below: R = -Cost min (p) = [-αT d (p)+βB(p)-γLo(p)],

[0062] Among them, T d (p), B(p), and Lo(p) represent the end-to-end delay, available bandwidth, and packet loss rate of path p between node i and node j; α, β, γ∈[0,1] are the weight coefficients of each optimization objective, and the weight values ​​are different when dealing with different business flows;

[0063] S62, initialize the DQN neural network, including preset learning rate The initial values ​​of the discount factor μ and the exploration rate ε; the DQN neural network is used to estimate the Q value corresponding to the action a of the reward function R; initialize the training pool, that is, create an experience replay buffer to store the experience gained by the agent's interaction with the environment, including the current network environment state information S t 、Action a t , reward r t , the next network environment status information S t+1 ;

[0064] S63, the agent interacts with the environment and obtains the current network environment status information S t , and according to the current network environment status information S t and exploration rate ε to select action a t , which is the weight value of the data flow transmission path, that is, the path is selected for data transmission; among them, the exploration rate ε is a hyperparameter with a value of (0,1), which represents the probability of the agent selecting a random action and is used to exploratoryally select non-optimal actions. If the random number is less than the exploration rate, the agent selects a random action; otherwise, the optimal action is selected according to the current Q value;

[0065] S64, the agent performs action a t , that is, a path p is selected to route the data packet, and the agent performs action a t Then get the corresponding reward r t , that is, the feedback of the network cost value, and at the same time S t Updated to S t+1 The experience value (s) gained by the agent interacting with the environment t ,a t ,r t ,s t+1 ) is stored in the DQN experience pool;

[0066] S65, in the DQN experience pool, the data priority screening module is deployed in the experience tuple of the experience pool, and the neural network collects learning sample batches based on data priority. include:

[0067] S65-1. The first batch of experience gained by the agent through interaction with the environment (s t ,a t ,rt ,s t+1 ) is directly input into the Q network, and then passes through the Q_target network to obtain the Q value prediction value Q′ of each action at time t;

[0068] The target network Q_target has the same structure as the main network Q, which is used to provide a relatively stable target Q value and solve the over-estimation problem of the DQN algorithm.

[0069] S65-2, select the corresponding optimal action a′ according to the Q′ value t , as follows: a′ t =argmaxQ(s t , a t θ t ),

[0070] Where θ represents the parameters of the Q network, which is used to adjust the weight of the Q network so that the Q network can more accurately estimate the Q value of each action, and t is a preset time unit;

[0071] S65-3, calculate and execute a′ t The corresponding reward value r′ t ;

[0072] S65-4, define the experience pool data priority function Pr(t): Pr(t)=(r′ t +Q′) λ ,

[0073] Among them, λ is a hyperparameter and λ∈(0,1), which is used to control the attenuation degree of the priority function to directly filter the data with lower priority and improve the efficiency of model training;

[0074] S65-5, the reward value r' of each experience data t Substitute the corresponding Q′ into the priority function formula to calculate the priority of each experience data and load it into the experience pool; based on the interaction between the agent and the environment, periodically update the priority of the data in the experience pool;

[0075] S66: According to step S65, the data priority screening module collects learning sample batches The neural network calculates the Q value corresponding to the batch of samples and then obtains the current state Action corresponding to the lower Q value and perform actions The reward obtained by the agent Then calculate the target Q value, where the Q value corresponding to the batch of samples is:

[0076] Where, f θRepresents the output function of the main network, which is a function determined by the neural network, and the action corresponding to the Q value

[0077] Based on the data flow after the clustering algorithm is clustered into giant elephant flow L g 、Elephant Flow L e and rat flow r ,action That is, the weight value of the path p planned for data stream transmission;

[0078] Execute an action After that, the agent obtains the reward value of the environment As follows:

[0079] Reward Value The larger it is, the smaller the network cost is, and the better the action performed is, that is, the better the path planning solution is;

[0080] Calculate the target Q value:

[0081] Among them, μ is the discount factor, which is used to balance the importance of current rewards and future rewards;

[0082] S67, update Q network parameters θ t :Define the loss function Loss(θ t ), use the stochastic gradient descent method SGD to minimize the difference between the Q value of the current state and the target Q value, and update the parameters of the Q network;

[0083] Update the target Q_target network parameters θ′: that is, regularly copy the parameters of the Q network to the target Q_target network,

[0084] S68, the agent updates the state to And reduce the exploration rate ε, repeat the above steps S63 to S68 until the data center network state determined by the agent is the termination state, that is, the mouse flow, elephant flow, and giant elephant flow clustered by various services in the network are forwarded.

[0085] Furthermore, the aforementioned step S7 is specifically as follows:

[0086] Define the weight value set We of the optimal path min , as shown in the following formula, the role of this set is to determine the nodes that path p passes through after the DQN algorithm outputs the optimal action a, that is, the path p planned for data flow transmission, and the SDN control layer manages the network and issues flow tables;

[0087] Furthermore, the weight value in the aforementioned step S8 is calculated according to the following formula:

[0088] Wherein, K is the total number of K paths calculated using the K shortest path algorithm based on the number of hops, and represents the flow transmitted on the kth path between nodes i and j.

[0089] Compared with the prior art, the beneficial technical effects of the present invention using the above technical solution are as follows:

[0090] 1. The present invention adopts SDN technology to separate the network control plane and data plane, realize dynamic adjustment and optimization of network traffic, and collect traffic demand and congestion status between each link by deploying a unified SDN controller, so as to realize flexible on-demand allocation of data center network resources, maximize the utilization rate of data center network resources, and improve network performance and QoS guarantee.

[0091] 2. The present invention improves the traditional k-means algorithm by using a traffic-based distribution probability density function to determine the initial cluster centers. By analyzing the data traffic, the traffic distribution probability density function is obtained. By determining the location of the initial cluster centers based on the traffic distribution probability density function, the cluster centers are more closely aligned with the data distribution, thereby improving the accuracy of the algorithm. This greatly avoids the problems of the traditional K-means algorithm that result in less robust data processing and less accurate classification results due to the randomness of the initial cluster center selection.

[0092] 3. This invention applies the DQN algorithm to calculate and optimize the link weights of each node in the entire network. Compared to traditional methods, this invention maintains a priority queue in the DQN experience pool. By defining a data priority function within the experience pool, data is stored in the queue according to priority. During DQN neural network training, data is extracted from the experience pool based on priority, rather than the random and uniform method used in traditional DQN networks. This creates batches of learning samples for the neural network, allowing the neural network to focus more on important data, improving its learning efficiency and algorithm performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] FIG1 is a diagram of an SDN network model of the present invention.

[0094] FIG2 is an overall flow chart of a network QoS assurance routing method based on SDN architecture of the present invention.

[0095] FIG3 is a diagram of a deep reinforcement learning algorithm model of the present invention. DETAILED DESCRIPTION

[0096] In order to better understand the technical content of the present invention, specific embodiments are given below in conjunction with the accompanying drawings.

[0097] Various aspects of the present invention are described herein with reference to the accompanying drawings, which show a number of illustrative embodiments. The embodiments of the present invention are not limited to those described in the accompanying drawings. It should be understood that the present invention can be implemented by any of the various concepts and embodiments described above, as well as the concepts and implementations described in detail below, because the concepts and embodiments disclosed herein are not limited to any particular implementation. In addition, some aspects disclosed herein may be used alone or in any appropriate combination with other aspects disclosed herein.

[0098] Referring to Figure 1, a network based on the SDN architecture of the present invention includes an application layer, a control layer and a data layer. The controller belonging to the SDN control layer is connected to the data layer switch. The traffic classification module and the QoS guarantee module are both deployed in the SDN controller, which is responsible for data center network traffic classification and routing optimization and other functions.

[0099] As shown in FIG2 , the present invention provides a network QoS guarantee routing method based on the SDN architecture, comprising the following steps: S1, the SDN controller periodically obtains data center network topology information according to a preset time t;

[0100] The topology information includes information such as the available bandwidth and end-to-end delay between nodes in the network, which is used for subsequent evaluation and calculation of routing strategies. Specifically, the SDN controller communicates with the switches in the network through the southbound OpenFlow protocol to obtain the connection relationship and link information of the switches, and periodically obtains and updates the global network topology information through the link discovery protocol LLDP. The SDN network contains x controllers and y switches, and the set of x controllers is C = {c1, c2, ..., c x}, and set the controller c1 as the central controller, and the set of y switches is S = {s1,s2,……,s y}.

[0101] Monitoring of link available bandwidth: First, set a fixed time interval t, and then use the SDN controller to periodically monitor the ports of each switching device in the network through the southbound OpenFlow protocol to obtain statistical information of each port. The statistical information includes the amount of data received and sent by each port, and then calculate the link available bandwidth, that is,

[0102] Among them, D r is the amount of data received by the port in the current time interval t, D′ t is the amount of data sent by the port in the previous time interval t, B c is the channel bandwidth defined based on Shannon's theory.

[0103] Link latency monitoring: When data is transmitted over an SDN network, the controller sends a Packet-Out message, instructing switch S1 to send the data stream to switch S2. However, when switch S2 does not have a flow table entry capable of processing the data stream, it responds by sending a Packet-In message to the SDN controller. Based on the sending and receiving times, the SDN controller can calculate the total round-trip time (T) for the message along this path (from the SDN controller to switch S1, then to switch S2, and finally from switch S2 back to the controller). tr .

[0104] The SDN controller then sends echorequest messages to S1 and S2 respectively, and switches S1 and S2 then send echo reply messages to the SDN controller respectively. The SDN controller obtains the time T1 and T2 from the SDN controller to switches S1 and S2 based on the time difference between sending the echorequest message and receiving the echoreply message, and then calculates the end-to-end delay T between switches S1 and S2. d :

[0105] S2. Based on the topology information, the data center network is abstracted into a directed graph G = (U, V, W). The data transmission path of the data center network is obtained through the directed graph and represented by a weight matrix. Where U represents the set of nodes u in the network, that is, u∈U; V represents the set of links v in the network. The connection between two nodes defines a link, that is, v∈V. The data transmission between the source node source node i and the node j in the directed graph G is calculated using the K shortest path algorithm based on the number of hops. There are K paths in total. Represents the flow transmitted on the kth path between nodes i and j. The weight value of the kth path when the data flow is transmitted between nodes i and j is defined as but

[0106] This value represents the weight of the kth path between nodes i and j at the initial time. The weight of the paths between nodes in the network at the initial time can be obtained, which is expressed as an N×N weight matrix Among them, w ij =R + , that is, a positive real number, indicating that there is a path connecting nodes i and j, that is, data is reachable; w ij =0, indicating that there is no path connecting i and j, i.e., the data is unreachable.

[0107] S3. The SDN controller receives the data stream from the source host and determines whether the destination node to which the data stream is to be sent is directly connected to the source node based on the directed graph. If so, the data packet is forwarded directly according to the network reachable path. Otherwise, step S4 is executed. The behavioral characteristics and statistical characteristics of network traffic show that traffic data of the same application type is often distributed in a relatively dense area.

[0108] S4. Use the improved k-means algorithm based on data distribution density to cluster the data streams. That is, first use the traffic distribution density function to determine the initial cluster center, and then cluster the data stream into data streams with various classification features, namely, mouse stream L r 、Elephant Flow L e , and Giant Elephant Flow L g ;

[0109] S41, for any node i in the network, calculate the Euclidean distance dis(l a ,l b ), as follows:

[0110] The data flow transmitted by node i in the network is represented by the vector set L={l i |l i ∈R p ,i=1,2,3,…,n}, n is the number of traffic transmitted in node i, p is the representation dimension of the traffic, l a and l b are any two data flows in the traffic set L;

[0111] Calculate the average Euclidean distance of all data flows of node i As follows:

[0112] in, The smaller the value, the smaller the average Euclidean distance of the data streams, which means the difference between the data streams is smaller, that is, the data streams of the node are more similar;

[0113] S42, calculate the data flow l in the traffic set L transmitted by node i a The data density den(l a ), as follows:

[0114] Among them, den(l a ) indicates that in data stream l a The transmission time t(l a ) the amount of data or the number of data packets transmitted, den(l a ) is used to measure the traffic intensity of the data flow;

[0115] Calculate the average data density of all data flows at node i As follows:

[0116] in, The larger the value, the greater the average data density of the data flow, which means that the proportion of data transmitted in node i is higher, that is, the more concentrated the traffic distribution is;

[0117] The average transmission time of n data streams transmitted by computing node i is As follows:

[0118] Compute the cluster center c corresponding to all data flows of node i i , as follows:

[0119] Among them, c i represents the common characteristics of all data flows of node i in terms of size, distance, density, etc.

[0120] S43, calculate the traffic distribution function F(l i ), as follows:

[0121] Where P() is the probability function,

[0122] Calculate the distribution density function f(l i ), as follows:

[0123] Among them, f(l i ) represents the flow rate l i The probability of being distributed within a certain interval;

[0124] S44. Repeat steps S41-S43 for all U nodes in the network according to the directed graph G = (U, V, W), obtain the distribution density function of the transmission flow of each node in the network, and use the set f all It is represented as follows: all ={f1, f2, ..., f i ,…f u}

[0125] The cluster mean center C corresponding to each node all , as follows: C all ={c1,c2,...,c i ,...,c u}

[0126] Average traffic distribution density function of all U nodes in the network As follows:

[0127] The average flow distribution density function It is used to measure the average value of the traffic density of each node in the network. By comparing with the value of this function, each node in the network is divided into high-density nodes f i h and low-density nodes f i l ;

[0128] S45, according to the high-density node f i h and low-density nodes f i l , and further get the following formula:

[0129] The cluster center corresponding to the high-density node is moved from C all Extracted from the set C as the preliminary value of the K-means clustering center, and h Indicated as follows: C h ={c1, c2, ..., c i …, c m}, i = 1, 2, 3…m, m < n

[0130] S46, from C h Find the data object c1 with the largest value as the first initial cluster center And change c1 from C h Delete in;

[0131] S47, then from C h Find the distance The farthest data object c2 is used as the second initial cluster center And change c2 from C h Delete in;

[0132] S48, from C h Find the distance and The farthest data object c3 is used as the third initial cluster center And change c3 from C h Delete in;

[0133] S49, assigning data streams to corresponding data clusters: using the initial cluster center set obtained in steps S46-S48 Cluster each data stream of each node in the network, calculate its distance from each initial cluster center, and assign it to the data cluster corresponding to the nearest cluster center;

[0134] S410, update cluster center: for each cluster, repeat steps S41-S42, calculate the average value of all data streams assigned to the cluster, that is, the center point of the data stream, and use the average value as the new cluster center

[0135] S411, repeat steps S44-S410 until the cluster center no longer changes, save the clustering results, and cluster the data flow into mouse flow L r 、Elephant Flow L e 、Elephant Flow L g .

[0136] Since rat flows account for the largest proportion in the data center network, elephant flows account for a smaller proportion, and giant elephant flows account for the smallest proportion, the improved k-means algorithm based on the probability density distribution of traffic is used to obtain the initial cluster center. When the cluster center no longer changes, it indicates that the traffic in each cluster has converged successfully. That is, the network traffic is successfully clustered into rat flows L by using the probability distribution as the premise for selecting the cluster center and the data volume as the clustering standard. r 、Elephant Flow L e 、Elephant Flow L g .

[0137] Because different services, such as giant, large, and small flows, have different deterministic QoS standards and differentiated requirements in terms of transmission bandwidth, latency, jitter, and transmission security, QoS constraints are established to ensure QoS for different services. The DQN neural network is then used to generate weights for each link in the data center network.

[0138] The goal of multi-constraint QoS is to select the path for network traffic that best meets the QoS requirements for the application type. Specifically, a path p(i, j) must be found between nodes i and j in the network G = (U, V, W) that meets the QoS requirements for the service and minimizes the total network transmission cost. Minimizing the total network transmission cost effectively avoids network congestion, transforming the complex and volatile routing problem of SDN-based network traffic into an optimization problem.

[0139] S5. Calculate the end-to-end delay, available bandwidth, and packet loss rate of the path, and then construct the QoS constraint of the data flow, that is, the minimum total transmission cost Cost. min (p);

[0140] The QoS constraints of a data flow are obtained as follows:

[0141] S51. Calculate the end-to-end delay T of path p(i, j) d (p), which includes the sum of the transmission delay, propagation delay and queuing delay of the data packet at all nodes on path p, as shown in the following formula:

[0142] T d (p) and available bandwidth B(p) are obtained in step S1 when the SDN control plane periodically obtains the data center network topology structure according to a certain time t, where T t (v) T c (v) T q (v) are the transmission delay, propagation delay, and queuing delay of link v in path p; the transmission delay depends on the size of the data packet and the transmission capacity of the link; the propagation delay depends on the physical distance of the link and the signal propagation speed; the queuing delay required for the data packet to wait for transmission in the queue of the network device depends on the network congestion and queue length.

[0143] S52. Calculate the available bandwidth B(p) of path p, as follows:

[0144] Where t is the time interval for the preset SDN controller to obtain the network topology information of the network data center; D r is the amount of data received by the port in the current time interval t; D′ t is the amount of data sent by the port in the previous time interval t, B c is the channel bandwidth defined based on Shannon's theory;

[0145] S53, the packet loss rate of path p, that is, the number of packets lost during transmission divided by the number of packets sent. The number of lost packets is the number of packets that failed to reach the destination during transmission on the link, and the number of sent packets is the total number of packets sent on the link;

[0146] Calculate the packet loss rate of path p as follows:

[0147] Among them, D t-all is the total amount of data sent by the port in the current time interval t; D r is the amount of data received by the port in the current time interval t, and the difference between the two is the packet loss;

[0148] S54, calculate the minimum transmission cost of path p min (p), as follows: Cost min (p) = [αT d (p)-βB(p)+γLo(p)]

[0149] Among them, α, β, γ are the weight coefficients of link cost, and the minimum cost Cost min (p) Taking into account the combined effects of link bandwidth, delay and packet loss rate, the minimum cost is controlled min(p) to achieve the best performance of network transmission.

[0150] Refer to Figure 3, S6, build a DQN neural network, including the Q network and the target Q_target network; define the state space, including network topology information, source nodes, destination nodes, and data flows of various classification features; define the action space A for the interaction between the agent and the environment as the weight value of the data flow selecting the path transmission between nodes, and the reward function R as the minimum total transmission cost Cost of each path in the network min The negative correlation is established, and the maximum Q value corresponding to the action a with the largest reward function R is used as the target to train the DQN neural network. During the training process, the experience replay mechanism and the target Q_target network are used to optimize the routing strategy to obtain a QoS guaranteed routing model.

[0151] The model input includes the source node, destination node, traffic classification characteristics, network topology information and QoS constraints, namely the minimum total transmission cost. min During the training process, the DQN algorithm uses the experience replay mechanism and target network technology to continuously optimize the routing strategy and improve the QoS guarantee capability, including the following sub-steps:

[0152] S61, mapping the parameters of the DQN neural network to the actual application scenario of the SDN network, including the state space S, action space A and reward function R;

[0153] (1) The state space S includes: network topology information, source nodes, destination nodes, and data flows of various classification features.

[0154] The source node and the destination node include information such as the location, hop count, and connection relationship of each node in the network. The traffic feature information is the traffic classification in the current network that the agent needs to understand, that is, the giant flow L obtained by clustering based on the improved k-means algorithm in step S4. g 、Elephant Flow L e and rat flow r The state variables in the network, such as network load, bandwidth utilization, and network throughput, are used to enable the agent to select an appropriate traffic scheduling strategy.

[0155] (2) Define the action space A of the interaction between the agent and the environment:

[0156] a(s t ) is the weight value for selecting the pth path between nodes i and j for data flow l There are K paths between nodes i and j. Data stream l is transmitted on the pth path between nodes i and j. The weight of the pth path is:

[0157] in represents the traffic transmitted on path p between nodes i and j;

[0158] (3) In order to select the optimal path that meets the service QoS requirements and avoid the congestion problem that often occurs in data centers, the optimization goal of this embodiment is to minimize the sum of the link weights through which the data stream is transmitted. At the same time, the goal of the DQN neural network is to find the maximum Q value corresponding to the action a that maximizes the reward function R. The reward function R is the negative correlation of the minimum transmission cost of each path in the network, as shown in the following formula: R = -Cost min (p) = [-αT d (p)+βB(p)-γLo(p)]

[0159] Among them, T d (p), B(p), Lo(p) represent the end-to-end delay, available bandwidth, and packet loss rate of path p between nodes i and j; α, β, γ∈[0,1] are the weight coefficients of each optimization objective, and the weight values ​​are different when dealing with different business flows;

[0160] S62, initialize the DQN neural network, including preset learning rate The initial values ​​of the discount factor μ and the exploration rate ε; the DQN neural network is used to estimate the Q value corresponding to the action a of the reward function R; initialize the training pool, that is, create an experience replay buffer to store the experience gained by the agent's interaction with the environment, including the current network environment state information S t 、Action a t , reward r t , the next network environment status information S t+1 , ensuring the stability of the DQN neural network;

[0161] S63, the agent interacts with the environment and obtains the current network environment status information S t , and according to the current network environment status information S t and exploration rate ε to select action a t , which is the weight value of the data flow transmission path, that is, the path is selected for data transmission; among them, the exploration rate is a hyperparameter with a value of (0, 1), which represents the probability of the agent selecting a random action and is used to exploratoryally select non-optimal actions. If the random number is less than the exploration rate, the agent selects a random action; otherwise, the optimal action is selected according to the current Q value;

[0162] S64, the agent performs action a t, that is, a path p is selected to route the data packet, and the agent performs action a t Then get the corresponding reward r t , that is, the feedback of the network cost value, and at the same time S t Updated to S t+1， The experience value (s) gained by the agent interacting with the environment t ,a t ,r t ,s t+1 ) is stored in the DQN experience pool;

[0163] S65, in the DQN experience pool, the data priority screening module is deployed in the experience tuple of the experience pool, and the neural network collects learning sample batches based on data priority. include:

[0164] S65-1. The first batch of experience gained by the agent through interaction with the environment (s t ,a t ,r t ,s t+1 ) is directly input into the Q network, and then passes through the Q_target network to obtain the Q value prediction value Q′ of each action at time t.

[0165] The target network Q_target has the same structure as the main network Q, which is used to provide a relatively stable target Q value and solve the over-estimation problem of the DQN algorithm.

[0166] S65-2, select the corresponding optimal action a′ according to the Q′ value t , as follows: a′ t =argmaxQ(s t ,a t θ t ),

[0167] Where θ represents the parameters of the Q network, which is used to adjust the weight of the Q network so that the Q network can more accurately estimate the Q value of each action, and t is a preset time unit;

[0168] S65-3, calculate and execute a′ t The corresponding reward value r′ t ;

[0169] S65-4, define the experience pool data priority function Pr(t):

[0170] Pr(t)=(r′ t +Q′) λ

[0171] Among them, λ is a hyperparameter and λ∈(0,1), which is used to control the attenuation degree of the priority function to directly filter the data with lower priority and improve the efficiency of model training;

[0172] S65-5, the reward value r' of each experience data t Substitute the corresponding Q′ into the formula of the priority function to calculate the priority of each experience data and load it into the experience pool; based on the interaction between the agent and the environment, the priority of the data in the experience pool is periodically updated.

[0173] S66: According to step S65, the data priority screening module collects learning sample batches The neural network calculates the Q value corresponding to the batch of samples and then obtains the current state Action corresponding to the lower Q value and perform actions The reward obtained by the agent Then calculate the target Q value, where the Q value corresponding to the batch of samples is:

[0174] Where, f θ Represents the output function of the main network, which is a function determined by the neural network, and the action corresponding to the Q value

[0175] .Based on the data flow being clustered into giant flow L after clustering algorithm g 、Elephant Flow L e and rat flow r ,action That is the weight value of the path p planned for data stream transmission.

[0176] Execute an action After that, the agent obtains the reward value of the environment As follows:

[0177] Reward Value The larger it is, the smaller the network cost is, and the better the action performed is, that is, the better the path planning solution is;

[0178] In this embodiment, the reward value Negatively correlated with network overhead, the physical meaning of this value is the evaluation index of the routing path planned for data flows between any two nodes in the network, which is based on the network QOS guarantee model based on the DQN algorithm deployed in the SDN control plane, combined with traffic business requirements and network resource overhead. The larger the value, the smaller the network cost, and the better the action executed, that is, the better the path planning solution. Conversely, the conclusion is the same.

[0179] Calculate the target Q value:

[0180] Among them, μ is the discount factor, which is used to balance the importance of current rewards and future rewards;

[0181] S67, update Q network parameters θ t :Define the loss function Loss(θ t ):

[0182] The loss function measures the neural network's prediction error—the difference between the current Q-value and the target Q-value. The loss function guides the update of the main network parameters. By calculating the gradient of the loss function with respect to the parameters, we can determine the direction and magnitude of the parameter update, gradually improving the network's prediction ability. This, in turn, brings the neural network's predictions closer to the true value, improving the agent's decision-making accuracy.

[0183] Use stochastic gradient descent (SGD) to minimize the difference between the current state Q value and the target Q value, and update the parameters of the Q network.

[0184] in, The learning rate is a hyperparameter used to control the step size or speed of neural network parameter updates, which determines the amplitude of the parameter movement along the gradient direction each time the parameter is updated; is the gradient of the loss function with respect to the parameters.

[0185] Update the target Q_target network parameters θ′: that is, regularly copy the parameters of the Q network to the target Q_target network, then

[0186] θ′ t+1 =τ×θ t+1 +(1-τ)θ′ t

[0187] Among them, τ is the parameter update rate, which controls the influence of the main network parameters on the target network parameters.

[0188] S68, the agent updates the state to In order to use the updated state to make decisions in the next training step, reduce the exploration rate ε, and reduce the probability of random exploration of the agent, repeat the above steps S63-S68 until the data center network state determined by the agent is the terminal state, that is, the mouse flow, elephant flow, and giant elephant flow clustered by various services in the network are forwarded.

[0189] S7. Using the QoS guarantee routing model, obtain the weight value of the optimal path for data flow transmission, obtain the optimal path for corresponding data flow transmission based on the weight value, and further determine all nodes that the transmission path passes through;

[0190] Define the weight value set We of the optimal path min , as shown in the following formula, the role of this set is to determine the nodes that path p passes through after the DQN algorithm outputs the optimal action a, that is, the path p planned for data flow transmission, so as to facilitate the SDN control layer to manage the network and issue flow tables;

[0191] in, Indicates whether the path p planned between node i and node j contains node z, The value is 1 or 0. If the value is 1, the node z is included. If the value is 0, the node z is not included.

[0192] S8. According to the Dijkstra algorithm, the node with the smallest weight value among all the nodes passed by the transmission path is selected as the routing node to determine the transmission path of the data stream.

[0193] Finally, the routers, routing protocols, and interfaces are configured, and the transmission node information corresponding to the optimal path selected in steps S7 and S8 is configured into a routing table and sent to the switch of the SDN data plane. The switch forwards data based on the sent path information, realizing route optimization and QoS guarantee for traffic transmission, ensuring the efficiency of traffic transmission and QoS requirements.

[0194] While the present invention has been described above with reference to preferred embodiments, this is not intended to limit the present invention. Persons skilled in the art will readily appreciate that various modifications and variations can be made without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A network QoS guarantee routing method based on SDN architecture, characterized in that: The steps include: S1, the SDN controller periodically obtains the data center network topology information according to the preset time t; S2. The data center network is abstracted into a directed graph based on the topology information. The reachable path for data transmission in the data center network is obtained through the directed graph and represented by a weight matrix. Specifically, the directed graph is G = (U, V, W), where U represents the set of nodes u in the network, i.e., u∈U; V represents the set of links v in the network, and the connection between two nodes defines a link, i.e., v∈V. Represents the weight matrix of the network path nodes at the initial time, where w ij =R + , that is, a positive real number, indicating that there is a path connecting nodes i and j, that is, the data is reachable; w ij =0, indicating that there is no path connecting i and j, i.e., the data is unreachable; S3, the SDN controller receives the data flow from the source host, and determines whether the destination node to which the data flow is to be sent is directly connected to the source node according to the directed graph. If yes, the data packet is forwarded directly according to the network reachable path, otherwise, step S4 is executed; S4, clustering the data streams using an improved k-means algorithm based on data distribution density, that is, first determining the initial cluster center using the traffic distribution density function, and then clustering the data streams into data streams containing various classification features, namely, mouse streams, elephant streams, and giant elephant streams; specifically including the following sub-steps: S41, for any node i in the network, calculate the Euclidean distance dis(l a ,l b ), as follows: Among them, the data flow transmitted by node i in the network is represented by the vector set L = {l i |l i ∈R p ,i=1,2,3,...,n}, n is the number of traffic transmitted in node i, p is the representation dimension of the traffic, l a and l b are any two data flows in the traffic set L; Calculate the average Euclidean distance of all data flows of node i As follows: in, The smaller the value, the smaller the average Euclidean distance of the data streams, which means the difference between the data streams is smaller, that is, the data streams of the node are more similar; S42, calculate the data flow l in the traffic set L transmitted by node i a The data density den(l a ), as follows: Among them, den(l a ) indicates that in data stream l a The transmission time t(l a ) is the amount of data or the number of packets transmitted within a period of time, den(l a ) is used to measure the traffic intensity of the data flow; Calculate the average data density of all data flows of node i As follows: in, The larger the value, the greater the average data density of the data flow, which means that the proportion of data transmitted in node i is higher, that is, the more concentrated the traffic distribution is; The average transmission time of n data streams transmitted by computing node i is As follows: Calculate the cluster center c corresponding to all data flows of node i i , as follows: Among them, c i Represents the common characteristics of all data flows of node i in terms of size, distance, and density; S43, calculate the traffic distribution function F(l i ), as follows: Where P() is the probability function; Calculate the distribution density function f(l i ), as follows: Among them, f(l i ) indicates flow rate l i The probability of being distributed within a certain interval; S44. Repeat steps S41-S43 for all U nodes in the network according to the directed graph G = (U, V, W), obtain the distribution density function of the transmission flow of each node in the network, and use the set f all It is expressed as follows: f all ={f1,f2,…,f i ,…f u }, The cluster mean center C corresponding to each node all As follows: C all ={c1,c2,…,c i ,…,c u }, Average traffic distribution density function of all U nodes in the network As follows: Average flow distribution density function It is used to measure the average value of the traffic density of each node in the network. By comparing with the value of this function, each node in the network is divided into high-density nodes f i h and low density nodes f i l ; S45, according to the high-density node f i h and low density nodes f i l , and further get the following formula: The cluster centers corresponding to the high-density nodes are moved from C all Extracted from the set C as the preliminary value of the K-means clustering center, and h Indicates as follows: C h ={c1,c2…,c i …,c m },i=1,2,3…m,m<n, S46, from C h Find the data object c1 with the largest value as the first initial cluster center And change c1 from C h Delete from; S47, then from C h Find the distance The farthest data object c2 is used as the second initial cluster center And c2 from C h Delete from; S48, from C h Find the distance and The farthest data object c3 is used as the third initial cluster center And change c3 from C h Delete from; S49, assigning data streams to corresponding data clusters: using the initial cluster center set obtained in steps S46-S48 Cluster each data stream of each node in the network, calculate its distance from each initial cluster center, and assign it to the data cluster corresponding to the nearest cluster center; S410, update cluster center: for each cluster, repeat steps S41-S42, calculate the average value of all data streams assigned to the cluster, that is, the center point of the data stream, and use the average value as the new cluster center S411, repeat steps S44-S410 until the cluster center does not change, save the clustering result, and cluster the data flow into mouse flow L r 、Elephant Flow L e , Giant Elephant Flow L g ; S5, calculate the end-to-end delay of the path, the available bandwidth of the path, and the packet loss rate of the path, and then construct the QoS constraint condition of the data flow, that is, the minimum total transmission cost Cost min ; S6. Construct a DQN neural network, including a Q network and a target Q_target network; define the state space, including network topology information, source nodes, destination nodes, and data streams of various classification features; define the action space A for the interaction between the agent and the environment as the weight value for the data stream to select the path transmission between nodes, and the reward function R as the negative correlation of the minimum total transmission cost of each path in the network. The DQN neural network is trained with the maximum Q value corresponding to the action a with the largest reward function R as the target. During the training process, the experience replay mechanism and the target Q_target network are used to optimize the routing strategy to obtain a QoS guaranteed routing model; S7, using the QoS guarantee routing model, obtain the weight value of the optimal path for data stream transmission, obtain the optimal path for corresponding data stream transmission based on the weight value, and further determine all nodes passed by the transmission path; S8. According to the Dijkstra algorithm, the node with the smallest weight value among all the nodes passed by the transmission path is selected as the routing node to determine the transmission path of the data stream.

2. According to a network QoS guarantee routing method based on SDN architecture according to claim 1, it is characterized in that: In step S1, the SDN controller communicates with the switches in the network through the southbound OpenFlow protocol to obtain the connection relationship and link information of the switches, and periodically obtains and updates the global network topology information through the link discovery protocol LLDP. The SDN network includes x controllers and y switches, and the set of x controllers is C = {c1, c2, ..., c x }, and set controller c1 as the central controller, and the set of y switches is S = {s1, s2, ..., s y }.

3. According to a network QoS guarantee routing method based on SDN architecture according to claim 2, it is characterized in that: In step S5, the QoS constraints of the data flow are obtained according to the following steps: S51. Calculate the end-to-end delay T of path p(i, j) d (p), as follows: Among them, T t (v) T c (v) T q (v) are the transmission delay, propagation delay and queuing delay of link v in path p respectively; S52. Calculate the available bandwidth B(p) of path p, as follows: Where t is the preset time interval for the SDN controller to obtain the network topology information of the network data center; D r is the amount of data received by the port in the current time interval t; D′ t is the amount of data sent by the port in the previous time interval t, B c is the channel bandwidth defined based on Shannon’s theory; S53, calculate the packet loss rate of path p: Among them, D t-all is the total amount of data sent by the port in the current time interval t; D r is the amount of data received by the port in the current time interval t, and the difference between the two is the packet loss amount; S54, calculate the minimum transmission cost Cost of path p min (p), as follows: Cost min (p)=[αT d (p)-βB(p)+γLo(p)] Among them, α, β, γ are the weight coefficients of link cost, and the minimum cost Cost min (p) It takes into account the combined effects of link bandwidth, latency, and packet loss rate, and controls the minimum cost. min (p) to achieve optimal performance of network transmission.

4. According to the SDN architecture-based network QoS guarantee routing method of claim 3, it is characterized in that: Step S6 includes the following sub-steps: S61, mapping the parameters of the DQN neural network to the actual application scenario of the SDN network, including the state space S, the action space A and the reward function R; The state space S includes network topology information, source nodes, destination nodes, and data streams of various classification features; the action space A that defines the interaction between the agent and the environment is as follows: a(s t ) is the weight value of data flow l choosing to transmit on the pth path between node i and node j There are K paths between node i and node j. Data stream l is transmitted on the pth path between node i and node j. The weight of the pth path is: in, represents the traffic transmitted on path p between node i and node j; The reward function R is the negative correlation of the minimum transmission cost of each path in the network, as shown below: R=-Cost min (p)=[-αT d (p)+βB(p)-γLo(p)] Among them, T d (p), B(p), Lo(p) represent the end-to-end delay, available bandwidth, and packet loss rate of path p between node i and node j; α, β, γ∈[0,1] are the weight coefficients of each optimization objective, and the weight values ​​are different when dealing with different business flows; S62, initialize the DQN neural network, including preset learning rate The initial values ​​of the discount factor μ and the exploration rate ε; the DQN neural network is used to estimate the Q value corresponding to the action a of the reward function R; initialize the training pool, that is, create an experience playback buffer to store the experience gained by the agent's interaction with the environment, including the current network environment state information S t 、Action a t , Reward t , the next network environment status information S t+1 ; S63, the agent interacts with the environment and obtains the current network environment status information S t , and according to the current network environment status information S t and exploration rate ε to select action a t , which is the weight value of the data flow transmission path, that is, the path is selected for data transmission; where the exploration rate ε is a hyperparameter with a value of (0,1), which represents the probability of the agent selecting a random action and is used to exploratoryally select non-optimal actions. If the random number is less than the exploration rate, the agent selects a random action; otherwise, the optimal action is selected according to the current Q value; S64, the agent performs action a t , that is, a path p is selected to route the data packet, and the agent performs action a t Then get the corresponding reward r t , that is, the feedback of the network cost value, and at the same time S t Updated to S t+1 , the experience value (s) gained by the agent interacting with the environment t ,a t ,r t ,s t+1 ) is stored in the DQN experience pool; S65, in the DQN experience pool, the data priority screening module is deployed in the experience tuple of the experience pool, and the neural network collects learning sample batches based on data priority. include: S65-1. The first batch of experience gained by the agent through interaction with the environment (s t ,a t ,r t ,s t+1 ) is directly input into the Q network, and then passes through the Q_target network to obtain the Q value prediction value Q′ of each action at time t; Among them, the target network Q_target has the same structure as the main network Q, which is used to provide a relatively stable target Q value and solve the over-estimation problem of the DQN algorithm; S65-2. Select the corresponding optimal action a' according to the Q' value t , as follows: a′ t argmaxQ(s). t ,a t θ t ) Among them, θ represents the parameters of the Q network, which is used to adjust the weight of the Q network so that the Q network can more accurately estimate the Q value of each action, and t is a preset unit time; S65-3, calculate and execute a′ t The corresponding reward value r′ t ; S65-4. Define the experience pool data priority function Pr(t): Pr(t)=(r′ t (+Q′) λ , Among them, λ is a hyperparameter and λ∈(0,1), which is used to control the attenuation degree of the priority function to directly filter the data with lower priority and improve the efficiency of model training; S65-5. Set the reward value r' of each experience data t Substitute the corresponding Q′ into the formula of the priority function, calculate the priority of each experience data, and load it into the experience pool; according to the interaction between the agent and the environment, periodically update the priority of the data in the experience pool; S66: According to step S65, the learning sample batches are collected based on the data priority screening module. The neural network calculates the Q value corresponding to the sample batch and then obtains the current state The action corresponding to the lower Q value and perform actions The reward obtained by the post-agent Then calculate the target Q value, where the Q value corresponding to the sample batch is: In the formula, f θ Represents the output function of the main network, which is a function determined by the neural network, and the action corresponding to the Q value Based on the data flow being clustered into giant elephant flow L after clustering algorithm g 、Elephant Flow L e And mouse flow r ,action That is, the weight value of the path p planned for data stream transmission; Execute an action After that, the agent obtains the reward value of the environment As follows: Reward Value The larger it is, the smaller the network cost is, and the better the action performed is, that is, the better the path planning solution is; Calculate the target Q value: Among them, μ is a discount factor, which is used to balance the importance of current rewards and future rewards; S67, update Q network parameters θ t :Define the loss function Loss(θ t ), use the stochastic gradient descent method SGD to minimize the difference between the current state Q value and the target Q value, update the parameters of the Q network; update the target Q_target network parameters θ′: that is, regularly copy the parameters of the Q network to the target Q_target network, S68, the agent updates the state to And reduce the exploration rate ε, repeat the above steps S63 to S68 until the intelligent agent determines that the data center network state is a termination state, that is, the forwarding of the mouse flow, elephant flow, and giant elephant flow clustered by various services in the network is completed.

5. According to the SDN architecture-based network QoS guarantee routing method of claim 4, it is characterized in that: Step S7 is specifically as follows: Define the weight value set We of the optimal path min , as shown below, the role of this set is to determine the nodes that path p passes through after the DQN algorithm outputs the optimal action a, that is, the path p planned for data flow transmission, and the SDN control layer manages the network and sends the flow table; 6. A network QoS guarantee routing method based on SDN architecture according to claim 5, characterized in that: The weight value in step S8 is calculated according to the following formula: Among them, K is the total number of K paths calculated using the K shortest path algorithm based on the number of hops. represents the flow transmitted on the kth path between nodes i and j.

Images