Flow load balancing method and device, electronic equipment and medium

Through the combination of SDN controller and reinforcement learning module, the network information of the drone group is used to dynamically adjust the traffic allocation strategy, which solves the problem that traditional load balancing methods cannot deal with traffic changes in the cloud network in real time, and realizes the efficient utilization of network resources and stable operation.

CN118945114APending Publication Date: 2024-11-12NANJING UNIV OF AERONAUTICS & ASTRONAUTICS +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411193288.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Traditional load balancing methods lack real-timeness and cannot effectively deal with link congestion caused by traffic changes in cloud networks, resulting in decreased network throughput and increased latency.

Method used

Using a combination of SDN controller and reinforcement learning module, the network status is monitored and predicted in real time through the network information of the drone group, dynamically adjust the traffic allocation strategy, reassign the network traffic path, and optimize traffic allocation using DDQN and CNN models to realize the network Real-time relief of congestion.

Benefits of technology

It improves network resource utilization, reduces network congestion, ensures stable network operation, and significantly improves the transmission delay and throughput of small streams.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118945114A_ABST
    Figure CN118945114A_ABST
Patent Text Reader

Abstract

The invention discloses a traffic load balancing method and device, electronic equipment and a medium, and the method comprises the steps: an SDN controller outputs a network state based on the network information of an unmanned plane group, so that a reinforcement learning module gives a traffic distribution strategy based on the network state, the traffic distribution strategy is used for redistribution of a target network traffic path of the unmanned aerial vehicle group; the SDN controller issues the traffic distribution strategy to the unmanned aerial vehicle group and then obtains the network information of the unmanned aerial vehicle group again; and the SDN controller outputs the network state again according to the network information of the unmanned aerial vehicle group after flow redistribution, so that the reinforcement learning module generates an award based on the network state, and the award is used for updating the reinforcement learning module. According to the traffic load balancing method provided by the invention, when the target network traffic path of the unmanned aerial vehicle group is congested, the target network traffic path is redistributed, so that the utilization rate of network resources is improved, the network congestion is reduced, and the stable operation of the network is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of communication networks, and in particular relates to a flow load balancing method, device, electronic equipment and medium. Background Art

[0002] With the continuous development of big data, cloud computing and modern network technology, the scale of data center network (DCN) continues to expand and the internal data traffic increases dramatically, which puts higher requirements on the bandwidth performance and network demand of DCN. In cloud networks, traffic can be divided into large flows and small flows: 90% of data flows belong to small flows. A few large flows will generate small flows, which will cause drastic changes in the network environment and eventually lead to link congestion. In addition, a large number of data packets are likely to occupy the cache queue of the switch, forming head-of-line blocking, which increases the delay of small flows. DCN usually uses equal-cost multi-path algorithms including ECMP, which evenly distributes each flow in the network to the path through a hash function, but the ECMP algorithm cannot sense whether there is congestion in the network. When there are large flows in the network, ECMP will distribute multiple large flows to the same link or path, causing or aggravating the network congestion, and significantly reducing the network throughput. The network environment changes rapidly, and traditional load balancing solutions lack real-time performance and are prone to network load.

[0003] Therefore, in view of the above technical problems, it is necessary to provide a traffic load balancing method, device, electronic device and medium. Summary of the invention

[0004] The purpose of the present invention is to provide a traffic load balancing method, device, electronic device and medium, which can solve the technical problems that traditional load balancing methods lack real-time performance and easily cause network load congestion.

[0005] In order to achieve the above object, a specific embodiment of the present invention provides a traffic load balancing method, the method comprising:

[0006] The SDN controller outputs a network state based on the network information of the drone swarm, so that the reinforcement learning module can provide a traffic allocation strategy based on the network state, and the traffic allocation strategy is used to reallocate the target network traffic path of the drone swarm;

[0007] The SDN controller sends the traffic distribution strategy to the drone group and then obtains the network information of the drone group again;

[0008] The SDN controller outputs the network status again according to the network information of the drone group after traffic redistribution so that the reinforcement learning module can generate rewards based on the network status, and the rewards are used to update the reinforcement learning module.

[0009] In one or more embodiments of the present invention, the method also includes: the SDN controller predicts the network status based on the network information output of the drone swarm, so that the reinforcement learning module provides a traffic pre-allocation strategy based on the predicted network status, and the traffic pre-allocation strategy is used to reallocate the target network traffic path of the drone swarm before congestion occurs.

[0010] In one or more embodiments of the present invention, the method further comprises:

[0011] The reinforcement learning module evaluates the reward obtained by the traffic distribution strategy in the current state through the Q value, the Q value is selected through the DDQN model, and the Q value is calculated through the CNN model;

[0012] The DDQN model includes a main network and a target network, wherein the main network is used to estimate the reward obtained by the allocation strategy, and the target network is used to select the allocation strategy;

[0013] In the CNN model, Q=(s, a; θ), where s is the state space, a is the allocation strategy, and θ represents the weight of the network.

[0014] In one or more embodiments of the present invention, the method further comprises:

[0015] The network status includes the port load and link bandwidth of the drone swarm switch. The state space of the network status is converted into a CNN image in the reinforcement learning module, and the CNN image is used to reflect the status of each switch in the drone swarm.

[0016] In one or more embodiments of the present invention, the traffic distribution strategy includes constructing a link list in the drone swarm, wherein the link list is used to transfer the target network traffic from the source address to the destination address, and the traffic distribution strategy corresponds to the target network traffic path, and the link list includes traffic paths from a single node and / or multiple nodes to a single node and / or multiple nodes.

[0017] In one or more embodiments of the present invention, the reward includes a large flow reward and a small flow reward, the evaluation parameters of the large flow reward include at least one of high throughput and low packet loss rate, and the evaluation parameters of the small flow reward include at least one of low latency and low packet loss rate.

[0018] In one or more embodiments of the present invention, the DDQN model further includes an experience pool, which is used to store state space s, allocation strategy a, rewards, and the next state space using allocation strategy a; the DDQN model samples the experience pool for updating.

[0019] In another aspect of the present invention, a traffic load balancing device is provided, the device comprising:

[0020] An allocation module, which is used for the SDN controller to output the network status based on the network information of the drone group, so that the reinforcement learning module can provide a traffic allocation strategy based on the network status, and the traffic allocation strategy is used to reallocate the target network traffic path of the drone group;

[0021] A sending module, used for the SDN controller to obtain the network information of the drone group again after sending the traffic distribution strategy to the drone group;

[0022] The update module is used for the SDN controller to output the network status again according to the network information of the drone group after the traffic is redistributed so that the reinforcement learning module can generate rewards based on the network status, and the rewards are used to update the reinforcement learning module.

[0023] In another aspect of the present invention, an electronic device is provided, comprising: at least one processor; and a memory, wherein the memory stores instructions, and when the instructions are executed by the at least one processor, the at least one processor executes a traffic load balancing method.

[0024] In another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the traffic load balancing method are implemented.

[0025] Compared with the prior art, the traffic load balancing method of the present invention can monitor the network status in real time and dynamically adjust the traffic distribution strategy to adapt to the dynamic changes of network traffic. When the target network traffic path of the drone swarm is congested, the target network traffic path is reallocated to improve network resource utilization, reduce network congestion, and ensure stable network operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0027] Figure 1 This is a schematic diagram of a network link structure according to an embodiment of the present invention;

[0028] Figure 2 This is a schematic diagram of a traffic load balancing method according to an embodiment of the present invention;

[0029] Figure 3 A flow chart of a traffic load balancing method according to an embodiment of the present invention;

[0030] Figure 4 It is a statistical diagram of the average throughput in the traffic load balancing method of the present invention and the prior art experiment;

[0031] Figure 5 It is a statistical diagram of the transmission delay FCT of small flow tasks in the traffic load balancing method of the present invention and the prior art experiment;

[0032] Figure 6 It is a statistical diagram of the number of unfinished scheduled flows in the expected time in the traffic load balancing method of the present invention and the prior art experiment;

[0033] Figure 7 It is a statistical diagram of the end-to-end delay of task transmission in the traffic load balancing method of the present invention and the prior art experiment;

[0034] Figure 8 This is a module diagram of a traffic load balancing device according to another embodiment of the present invention;

[0035] Fig. 9 This is a hardware structure diagram of a computing device according to another embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0037] like Figure 1 As shown, the application scenario of the traffic load balancing method of the present invention is introduced, that is, based on the end-edge-cloud structure of the drone, including the cloud layer, the edge layer and the end layer. In the cloud-edge-end network, the traffic includes large flows and small flows, of which 90% of the data flows belong to small flows. A small number of large flows will generate small flows, which will cause drastic changes in the network environment and eventually lead to link congestion. The traffic load balancing method of the present invention optimizes the elastic scheduling of resources and traffic load balancing in the cloud-edge-end network based on the end-edge-cloud structure of the drone and the prediction of the network links of the end-edge-cloud structure of the drone.

[0038] like Figures 2-3 As shown, a specific implementation of the traffic load balancing method is introduced. In this implementation, the method specifically includes the following steps:

[0039] S101, the SDN controller outputs the network status based on the network information of the drone group, so that the reinforcement learning module can give a traffic allocation strategy based on the network status. The traffic allocation strategy is used to reallocate the target network traffic path of the drone group.

[0040] In this embodiment, based on the current network environment, the SDN controller outputs the network status according to the network information of the drone group, and the network status includes the port load and link bandwidth of the drone group switch; in this embodiment, the SDN controller includes a network monitoring module, a link prediction module and a flow table delivery module, wherein the network monitoring module is used to monitor the network information of the drone group, the SDN controller outputs the network status according to the network information, and the learning reinforcement module converts it into a CNN image according to the network status. In this embodiment, the network status is output to the learning reinforcement module, and the state space of the network status is represented as a CNN image. The CNN image is used to reflect the status of each switch in the drone group, wherein the port load and link bandwidth information of each switch are mapped to a two-dimensional pixel matrix, and the network status is represented by a two-dimensional array, and each row in the pixel matrix represents the status of each switch at a specific time; each column represents the port load and link bandwidth utilization of the switch, and the network status space can be expressed as:

[0041]

[0042] Among them, S is the network state space; is the port load of the i-th switch at time t; is the link bandwidth utilization of the ith switch at time t, with a value range of (0, 1), and n is the number of switches in the drone swarm. In this embodiment, after the network state space is converted into a CNN image, the state of each switch in the network can be intuitively reflected and visualized. By analyzing the network state, we can better understand the operation of the network, discover problems in the network in a timely manner, and adjust and optimize it in real time.

[0043] In one embodiment, the reinforcement learning module gives a traffic distribution strategy based on the network state. The traffic distribution strategy for the drone swarm data flow execution includes: constructing a link list in the drone swarm, the link list is used to transfer the target network traffic from the source address to the destination address, the traffic distribution strategy corresponds to the target network traffic path, the link list includes a single node and / or multiple nodes to a single node and / or multiple nodes. In this embodiment, a transmission path of a single node is selected from the drone network and is represented as a subset apn, so the set of all traffic distribution strategies can be expressed as:

[0044] A=[ap1,ap2,…,ap n ];

[0045] Wherein, A represents the set of all traffic transmission paths. When traffic congestion occurs in the target network group, a corresponding path is selected from the set A of traffic transmission paths to alleviate the congestion.

[0046] In one embodiment, the SDN controller also includes a link prediction module, which outputs a predicted network status based on the network information of the drone swarm, so that the reinforcement learning module can provide a traffic pre-allocation strategy based on the predicted network status, and the traffic pre-allocation strategy is used to reallocate the target network traffic path before the target network traffic path of the drone swarm is congested; wherein the link prediction module identifies the network links that may be congested based on the current network status. In this embodiment, a capacity threshold can be preset according to the corresponding network link. When the corresponding network link traffic exceeds the preset capacity threshold, it can be considered that the corresponding link will be congested.

[0047] When the target network link is congested and / or it is predicted that the target network link is about to be congested, a traffic path is selected from the traffic distribution strategy set A to minimize the congestion in the network. In order to redistribute the traffic, a new traffic transmission path is first determined. The selection of the new traffic transmission path is based on whether the bandwidth of the new traffic transmission path is sufficient and whether it can carry the redistributed traffic to avoid congestion on the new path again. After determining the new traffic transmission path, part of the traffic is transferred from the congested link to the new traffic transmission path. This process can be achieved by adjusting the routing table to ensure that the data packets can be forwarded along the new traffic transmission path.

[0048] S102, the SDN controller sends the traffic distribution strategy to the drone group and obtains the network information of the drone group again;

[0049] In this embodiment, after the traffic distribution strategy is sent to the drone group, the SDN controller obtains new network information of the drone group through the network monitoring module after a preset time value, outputs the network status based on the new network information, and converts the state space of the network status into a CNN image, which reflects the status of each switch in the network on the one hand, and the effect of the traffic distribution strategy on the other hand.

[0050] S103, the SDN controller outputs the network status again according to the network information of the drone group after traffic redistribution so that the reinforcement learning module can generate rewards based on the network status, and the rewards are used to update the reinforcement learning module.

[0051] In one embodiment, the reinforcement learning module evaluates the reward obtained by the traffic allocation strategy in the current state through the Q value, the Q value is selected through the DDQN model, and the Q value is calculated through the CNN model; the DDQN model includes a main network and a target network, the main network is used to estimate the reward obtained by the allocation strategy, and the target network is used to select the allocation strategy; in the CNN model, Q = (s, a; θ); wherein s is the state space, a is the allocation strategy, and θ represents the weight of the network.

[0052] The CNN convolutional neural network calculates the Q value, Q = (s, a; 0). Where s represents the state space, a represents the allocation strategy, and 0 represents the parameters of the Q network. In this embodiment, the CNN is trained by minimizing the loss, and the loss function satisfies:

[0053] L0=(R+γQ(s',arg maxQ(s',a';θ);θ - )-Q(s,a;θ)) 2 ;

[0054] Where: L0 loss function; R is the reward obtained after taking action a in the current state; γ is the discount factor; s′ is the next state space after taking allocation strategy a; a′ is the allocation strategy in the next state space; θ - The parameters of the target network; arg maxQ(s′,a′;θ) is the optimal action in the next state; Q(s′,arg maxQ(s′,a′;θ); θ - ) is the target Q value.

[0055] In this embodiment, the SDN controller designs a reward function for the characteristics of large flows and small flows in the target network group according to the traffic distribution strategy given by the learning reinforcement module to generate rewards. The evaluation parameters of the large flow reward include at least one of high throughput and low packet loss rate, and the evaluation parameters of the small flow reward include at least one of low latency and low packet loss rate. For large flows, high throughput and low packet loss rate need to be obtained, and the reward function satisfies:

[0056] R L =α L H L,A +β L (1-L L,A );

[0057] Among them, R L is the reward function H of the large flow L,A is the average throughput of the large flow; the average packet loss rate of the LA large flow; α L , β L , the weight coefficient of the small flow reward function, and α L +β L =1.

[0058] For small flows, low latency and low packet loss rate are required, and the reward function satisfies:

[0059] R N =α N (1-t N,A )+β N (1-L N,A ):

[0060] Among them, R N is the reward function of the small flow, t N,A is the average round trip delay of the small flow; L N,A is the average packet loss rate of small flows; α N , β N , the weight coefficient of the small flow reward function, and α N +β N =1.

[0061] In a further embodiment, the DDQN model also includes an experience pool, which is used to store state space s, allocation strategy a, rewards, and the next state space using allocation strategy a; the DDQN model samples the experience pool for updating to further improve the accuracy and efficiency of traffic load balancing.

[0062] The following is an experimental test of the traffic load balancing method provided by the present invention:

[0063] The present invention uses a flow-level traffic scheduling simulator based on a Fat-Tree structure, sets the link bandwidth to 1 Gbps, generates traffic based on DCTCP and then schedules it, and finally compares the algorithm performance of the throughput and traffic delay of each load balancing algorithm.

[0064] In order to verify the effectiveness of the traffic load balancing method of the present invention, a traffic generator is constructed in this experiment. There are mainly two types of traffic communication modes, one to one and / or many to many. One to one means that a single host initiates traffic communication to a single host, which is mainly a random large flow, while many to many means that from the perspective of a host, traffic is transmitted to all hosts in the topology structure, where mainly small flows are transmitted.

[0065] The present invention generates three groups of traffic with similar sizes and rates, so that the results of traffic simulation are repeatable. In order to give the links in the topology structure equivalent loads to reflect the effect of the load balancing algorithm, in the traffic used in the experiment, the number of back traffic is about 25,000, the total number of query traffic is about 200,000, the size of the back traffic is about 3e10, the size of the total query traffic is 0.7e10, the number of total traffic is about 220,000, and the average bandwidth of each link exceeds 0.8 Gbps, which can reflect a certain scheduling performance of the load balancing algorithm.

[0066] Performance comparison of load balancing algorithms: In this experiment, the performance of each load balancing algorithm is evaluated mainly through four indicators, namely average throughput, small flow FCT, the number of scheduled flows that are not completed within the expected time, and the end-to-end delay of the flows.

[0067] like Figure 4 As shown, the average throughput uses the experimental results of multiple simulations. Assuming there are n servers and m simulations are performed, the average throughput is shown in the figure below. The unit is generally expressed in bits / second or bytes / second. In this experiment, Mbps is used, that is, 10e6 bits per second. The larger the average throughput, the better the utilization of the network bandwidth and the better the performance of the load balancing algorithm.

[0068]

[0069] TPS j is the throughput of each simulation. The throughput here is calculated based on the throughput within the expected completion time, where TPS j The calculation formula is as follows.

[0070]

[0071] Where T j is the throughput of the jth server.

[0072] like Figure 5 As shown in the figure, the small flow task transmission delay FCT: This indicator mainly reflects the transmission status of the small flow in the experiment. Since a query traffic data is specially set in the generated file, its size is fixed to 20480, i.e. 20kb. Since it is often the large flow that causes network congestion and small flow congestion in load balancing, the small flow transmission time can also well reflect the network congestion. Assuming that the simulation experiment is carried out m times, the calculation formula of FCT is as follows.

[0073]

[0074] The FCT jIt represents the transmission time of the kth small flow.

[0075] Assume that in a simulation experiment, the average transmission delay of the small flow tasks received by each server is t, and there are n servers in total, then FCT j The calculation formula is as follows.

[0076]

[0077] The smaller the FCT value of the small flow is, the better the performance of the load balancing algorithm is.

[0078] like Figure 6 As shown in the figure, the number of unscheduled flows within the expected time: Since the expected minimum completion time of the traffic generated during traffic generation is 10s, as long as we record how many flows are still not scheduled in the simulator after 10s, it can reflect the performance of various load balancing algorithms to a certain extent. The faster the number of unfinished traffic scheduling converges over time, the better the performance of the load balancing algorithm.

[0079] like Figure 7 As shown, the end-to-end delay of task transmission: Since the simulator is task-driven, the smaller the end-to-end delay of the traffic using the task, the better the traffic transmission performance. The average delay of all traffic is used here. The smaller the end-to-end delay of the total task traffic, the better the performance of the load balancing algorithm.

[0080] After adopting the traffic and experimental performance indicators generated in the previous two sections, three groups of experiments were conducted. The experimental objects were ECMP, WCMP, FlowBender and four load balancing methods of the present invention, and a control group that did not use load balancing to randomly select paths.

[0081] Under the condition of equivalent traffic load, the throughput of the algorithm of the present invention is comparable to that of the most outstanding FlowBender algorithm among these algorithms, and the transmission delay performance of small flows is better. In terms of throughput, the average throughput of the WCMP load balancing algorithm and the FlowBender algorithm is close to 800Mbps, which is close to the link bandwidth of 1Gbps, and the performance is good. However, in terms of the number of flows that are not completed within the expected time, when the minimum predicted completion time is 10s, all load balancing algorithms have not completed traffic scheduling. Among them, the random selection path algorithm that does not use the load balancing algorithm has reached nearly 2,000 flows that have not been allocated, and the ECMP algorithm also has more than 800 flow data to be allocated. After 10s, it is obvious that the convergence speed of the FlowBender algorithm is faster than that of the ECMP algorithm. In terms of the transmission delay of the task, the gap between Flowbender and the improved algorithm and the WCMP algorithm is not large, but in terms of the average transmission delay of small flows that reflects the transmission performance of small flows, the performance of the FlowBender algorithm is not very good, and the present invention has made significant progress in the transmission of small flows compared to the original algorithm.

[0082] like Figure 8 As shown, a device for implementing a traffic load balancing method provided in a specific embodiment of the present invention is introduced.

[0083] In the embodiment of the present invention, the provided traffic load balancing device includes a distribution module 201 , a sending module 202 and an updating module 203 .

[0084] Allocation module 201, used for the SDN controller to output the network status based on the network information of the drone group, so that the reinforcement learning module can provide a traffic allocation strategy based on the network status, and the traffic allocation strategy is used to reallocate the target network traffic path of the drone group;

[0085] The sending module 202 is used for the SDN controller to obtain the network information of the drone group again after sending the traffic distribution strategy to the drone group;

[0086] The update module 203 is used for the SDN controller to output the network status again according to the network information of the drone group after the traffic is redistributed so that the reinforcement learning module can generate rewards based on the network status, and the rewards are used to update the reinforcement learning module.

[0087] Fig. 9 According to the embodiment of this specification, a hardware structure diagram of a computing device 30 for traffic load balancing is shown. Fig. 9As shown, the computing device 30 may include at least one processor 301, a memory 302 (e.g., a non-volatile memory), a memory 303, and a communication interface 304, and the at least one processor 301, the memory 302, the memory 303, and the communication interface 304 are connected together via a bus 305. At least one processor 301 executes at least one computer-readable instruction stored or encoded in the memory 302.

[0088] It should be understood that the computer executable instructions stored in the memory 302, when executed, cause at least one processor 301 to perform the above combined operations in various embodiments of the present specification. Figure 1 Various operations and functions described for 8.

[0089] In the embodiments of the present specification, the computing device 30 may include, but is not limited to, a personal computer, a server computer, a workstation, a desktop computer, a laptop computer, a notebook computer, a mobile computing device, a smart phone, a tablet computer, a cellular phone, a personal digital assistant (PDA), a handheld device, a messaging device, a wearable computing device, a consumer electronic device, and the like.

[0090] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0091] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0092] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0093] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0094] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0095] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A traffic load balancing method, characterized in that: The method comprises: The SDN controller outputs a network state based on the network information of the drone swarm, so that the reinforcement learning module can provide a traffic allocation strategy based on the network state, and the traffic allocation strategy is used to reallocate the target network traffic path of the drone swarm; The SDN controller sends the traffic distribution strategy to the drone group and then obtains the network information of the drone group again; The SDN controller outputs the network status again according to the network information of the drone group after traffic redistribution so that the reinforcement learning module can generate rewards based on the network status, and the rewards are used to update the reinforcement learning module.

2. The traffic load balancing method according to claim 1, characterized in that: The method also includes: the SDN controller predicts the network status based on the network information output of the drone group, so that the reinforcement learning module provides a traffic pre-allocation strategy based on the predicted network status, and the traffic pre-allocation strategy is used to reallocate the target network traffic path of the drone group before congestion occurs.

3. The traffic load balancing method according to claim 1, characterized in that: The method further comprises: The reinforcement learning module evaluates the reward obtained by the traffic distribution strategy in the current state through the Q value, the Q value is selected through the DDQN model, and the Q value is calculated through the CNN model; The DDQN model includes a main network and a target network, wherein the main network is used to estimate the reward obtained by the allocation strategy, and the target network is used to select the allocation strategy; In the CNN model, Q=(s, a; θ), where s is the state space, a is the allocation strategy, and θ represents the weight of the network.

4. The traffic load balancing method according to claim 3, characterized in that: The method further comprises: The network status includes the port load and link bandwidth of the drone swarm switch. The state space of the network status is converted into a CNN image in the reinforcement learning module, and the CNN image is used to reflect the status of each switch in the drone swarm.

5. The traffic load balancing method according to claim 3, characterized in that: The traffic distribution strategy includes constructing a link list in the drone swarm, and the link list is used to transfer the target network traffic from the source address to the destination address. The traffic distribution strategy corresponds to the target network traffic path, and the link list includes traffic paths from a single node and / or multiple nodes to a single node and / or multiple nodes.

6. The traffic load balancing method according to claim 3, characterized in that: The reward includes a large flow reward and a small flow reward, the evaluation parameter of the large flow reward includes at least one of high throughput and low packet loss rate, and the evaluation parameter of the small flow reward includes at least one of low latency and low packet loss rate.

7. The traffic load balancing method according to claim 3, characterized in that: The DDQN model also includes an experience pool, which is used to store state space s, allocation strategy a, rewards, and the next state space using allocation strategy a; the DDQN model samples the experience pool for updating.

8. A traffic load balancing device, characterized in that: The device comprises: An allocation module, which is used for the SDN controller to output the network status based on the network information of the drone group, so that the reinforcement learning module can provide a traffic allocation strategy based on the network status, and the traffic allocation strategy is used to reallocate the target network traffic path of the drone group; A sending module, used for the SDN controller to obtain the network information of the drone group again after sending the traffic distribution strategy to the drone group; An update module is used for the SDN controller to output the network status again according to the network information of the drone group after traffic redistribution so that the reinforcement learning module can generate rewards based on the network status, and the rewards are used to update the update of the reinforcement learning module.

9. An electronic device, characterized in that: include: at least one processor; and a memory storing instructions, which, when executed by the at least one processor, enable the at least one processor to execute the traffic load balancing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the traffic load balancing method according to any one of claims 1 to 7 are implemented.