Unmanned aerial vehicle swarm integrated space-ground communication routing method, electronic equipment and system

Abstract

The present invention discloses a method, electronic device and system for integrated communication routing of a drone swarm between the sky and the earth. The method includes the following steps: obtaining the state of the drone swarm; establishing a reward function for the state of the drone swarm and the communication action; establishing a strategy optimization function according to the reward function; updating and iterating the communication action of the drone swarm according to the strategy optimization function to obtain the optimal communication action routing strategy of the drone swarm; and sending communication instructions to the drone swarm. The present invention updates and iterates the communication action of the drone swarm according to the strategy optimization function to obtain the optimal communication action routing strategy of the drone swarm, thereby obtaining the drone swarm communication action with low communication energy consumption and good communication quality, thereby reducing communication energy consumption while ensuring communication quality, improving the endurance of the drone swarm, and ensuring the communication quality of the drones; avoiding damage to the ground power system and paralysis of the ground communication network to cause trouble to the data transmission of the drone swarm.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle communication technology, and in particular to a ground-ground integrated communication routing method, electronic equipment and system for a group of unmanned aerial vehicles. Background Art

[0002] Inspection of power transmission lines during disasters is essential for identifying and diagnosing grid faults and is a key factor in quickly restoring power services. The increasing use of drones for power line inspections offers a safer and more cost-effective alternative to traditional inspections and manned helicopters. However, two major factors limit the efficiency of drone inspections: energy and communication. Limited by battery capacity, drones can typically operate for about 30 minutes when fully loaded, which not only requires frequent charging but also limits the coverage of drones, thereby reducing the overall efficiency of power line inspections. Communication constraints also play a key role, with limited communication range restricting the flight area of ​​drones. In addition, energy consumption during communication further shortens the operating time of drones. In extreme disaster situations, damaged power systems and paralyzed ground communication networks exacerbate these limitations, severely hindering the inspection capabilities of drones.

[0003] A flying ad hoc network is a network of multiple drones that offers significant advantages over a single drone, including maintaining communications in disaster scenarios, expanding the range of operations, and enhancing the resilience of drone swarms. However, a flying ad hoc network cannot solve the problem of obstructed real-time transmission of inspection video data due to interruptions in ground communication networks. Satellite Internet is a new type of global network infrastructure that uses network, aerospace, communication, and data technologies to provide a global coverage, comprehensive and continuous communication network. The main use of satellite Internet in disaster scenarios lies in its robustness and resilience. Therefore, satellite Internet can be combined with a flying ad hoc network to form an integrated satellite-ground network to achieve drone power line inspections in extreme disaster situations. The integrated space-ground network provides powerful communication capabilities, but also leads to increased communication energy consumption. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes a ground-ground integrated communication routing method for a drone swarm.

[0005] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0006] The method for integrating ground-ground communication routing of a drone swarm comprises the following steps: obtaining the state of the drone swarm;

[0007] A reward function is established regarding the state and communication action of a drone swarm, wherein the drone transmits data to other nearby drones or ground satellite terminals according to the communication action, and the ground satellite terminal is used to communicate with a satellite; a policy optimization function is established according to the reward function, wherein the reward function is associated with communication energy consumption and communication delay, and the reward function is negatively correlated with communication energy consumption and communication delay; the communication action of the drone swarm is updated and iterated according to the policy optimization function to obtain the optimal communication action routing strategy of the drone swarm; and a communication instruction is sent to the drone swarm, wherein the communication instruction includes the optimal communication action routing strategy.

[0008] Furthermore, the reward function is: ; is the reward function value of the entire drone swarm in the tth time period; is the sum of the communication path delays of the entire drone swarm in the tth time period; is the energy imbalance of the entire drone swarm in the tth time period; is the sum of the communication delay jitter of the entire drone group in the tth time period; is the communication energy consumption of the entire drone group in the tth time period; , , and They are delay weight coefficient, energy imbalance weight coefficient, delay jitter weight coefficient and communication energy consumption weight coefficient respectively.

[0009] Furthermore, the total delay of the communication path is Calculated according to the following formula: ; ; is the tth time period The path delay between the UAV and the ground satellite terminal, is the tth time period The delay of the i-th communication transmission path of the UAV; is the total number of drones in the drone swarm; For the The total number of segments of the communication transmission path of the UAV.

[0010] Furthermore, the communication delay jitter sum Calculated according to the following formula: ; ; is the tth time period Delay jitter of the drone; is the t-1th time period The path delay between the UAV and the ground satellite terminal.

[0011] Furthermore, the energy imbalance It can be calculated by the following formula: ; is the energy value of the drone with the largest remaining energy in the drone swarm after the end of the tth time period, It is the energy value of the drone with the smallest remaining energy in the drone swarm after the end of the tth time period.

[0012] Furthermore, the strategy optimization function is ; is the strategy parameter, is the state of the drone swarm in the tth time period; represents the expectation of all future returns based on the current strategy starting from the tth time period; To find the minimum function; is a range-limiting function; The clipping threshold for policy updates; is the importance sampling ratio; is the advantage function of the drone swarm in the tth time period.

[0013] Furthermore, the advantage function The specific calculation formula is as follows: ; is the reward weight factor; The status is The expected total return value at .

[0014] Furthermore, when the distance between the drone and other neighboring drones is less than a first preset communication distance, the drone can choose to transmit data to other neighboring drones.

[0015] Furthermore, when the distance between the UAV and the ground satellite terminal is less than a second preset communication distance, the UAV can choose to transmit data to the ground satellite terminal.

[0016] The present invention also proposes an electronic device, including a processor and a memory, wherein the memory is used to store program code and transmit the program code to the processor; the processor is used to execute the unmanned aerial vehicle swarm integrated space-ground communication routing method according to the instructions in the program code.

[0017] The present invention also proposes a space-ground integrated communication routing system for a drone swarm, comprising: a state acquisition module for obtaining the state of a drone swarm; a reward function establishment module for establishing a reward function regarding the state of a drone swarm and communication actions, wherein the drone transmits data to other neighboring drones or ground satellite terminals according to the communication actions; a strategy optimization function establishment module for establishing a strategy optimization function according to the reward function; a strategy update optimization module for updating and iterating the communication actions of the drone swarm according to the strategy optimization function to obtain the optimal communication action routing strategy for the drone swarm; and an instruction sending module for sending communication instructions to the drone swarm.

[0018] The present invention has the following beneficial effects:

[0019] By setting a reward function, and the reward function is negatively correlated with communication energy consumption and communication delay, a basis is provided for the optimization target of the subsequent strategy optimization function, so that the optimization target is low communication energy consumption and good communication quality. By updating and iterating the communication actions of the drone cluster according to the strategy optimization function, the optimal communication action routing strategy of the drone cluster is obtained, so as to obtain the communication actions of the drone cluster with low communication energy consumption and good communication quality, thereby reducing communication energy consumption while ensuring communication quality, improving the endurance of the drone cluster, and ensuring the communication quality of the drones; the drone cluster can transmit data to the ground satellite terminal, and the ground satellite terminal transmits the data by communicating with the satellite, thereby avoiding damage to the ground power system and paralysis of the ground communication network to cause trouble to the data transmission of the drone cluster.

[0020] In addition to the above-described purposes, features and advantages, the present invention has other purposes, features and advantages. The present invention will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings constituting a part of this application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0022] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0023] Figure 2 It is a schematic diagram of the overall relevant equipment of the present invention;

[0024] Figure 3 is a communication strategy of an embodiment of the present invention;

[0025] Figure 4 It is a communication strategy of another embodiment of the present invention. DETAILED DESCRIPTION

[0026] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0029] In addition, the descriptions of "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0030] Please refer to Figure 1 , a preferred embodiment of the present invention provides a method for integrated communication routing of a drone swarm, including the following steps: obtaining the state of the drone swarm; establishing a reward function for the state of the drone swarm and the communication action, wherein the drone transmits data to other neighboring drones or ground satellite terminals according to the communication action, and the ground satellite terminal is used to communicate with the satellite; establishing a strategy optimization function according to the reward function; updating and iterating the communication action of the drone swarm according to the strategy optimization function to obtain the optimal communication action routing strategy of the drone swarm; sending a communication instruction to the drone swarm, wherein the communication instruction includes the optimal communication action routing strategy, and the drone swarm communicates according to the communication action routing strategy contained in the communication instruction, wherein the communication action is that the direct communication object specifically selected by the drone is the drone or the ground satellite terminal. Different communication actions, i.e. different communication object selections, will affect the communication energy consumption and communication delay, so it is necessary to optimize the communication action to obtain the optimal communication action routing strategy of the drone swarm to reduce energy consumption and delay, and improve the stability and endurance of the drone when working.

[0031] The unmanned aerial vehicle swarm space-ground integrated communication routing method provided by the present invention provides a basis for the optimization target of the subsequent strategy optimization function by setting a reward function, and the reward function is negatively correlated with the communication energy consumption and the communication delay, so that the optimization target is low communication energy consumption and good communication quality, and the communication action of the unmanned aerial vehicle swarm is updated and iterated according to the strategy optimization function to obtain the optimal communication action routing strategy of the unmanned aerial vehicle swarm, thereby obtaining the unmanned aerial vehicle swarm communication action with low communication energy consumption and good communication quality, thereby reducing the communication energy consumption while ensuring the communication quality, improving the endurance of the unmanned aerial vehicle swarm, and ensuring the communication quality of the unmanned aerial vehicle; the unmanned aerial vehicle swarm can transmit data to the ground satellite terminal, and the ground satellite terminal transmits the data by communicating with the satellite, thereby avoiding damage to the ground power system and paralysis of the ground communication network to cause trouble to the data transmission of the unmanned aerial vehicle swarm.

[0032] Usually, the path planning of the drone swarm has been preset. The present invention does not plan the path of the drone swarm. The present invention optimizes the communication routing action of the drone swarm in the preset path planning of the drone swarm. The drone swarm can fly according to the preset path.

[0033] In order to better represent the status of the drone fleet, we can first perform geometric modeling on the power lines and build a ground-ground integrated network architecture:

[0034] Assume that a power line has The position of each tower can be expressed by three-dimensional coordinates:

[0035] .

[0036] in, For the The coordinates of the power tower.

[0037] Define a connectivity matrix , which is used to indicate which towers are connected by power lines. is a N matrix, where:

[0038]

[0039] if , then the drone needs to fly to inspect the power lines connecting the two towers; if , you can skip this connection.

[0040] For electric towers with power lines connected and , electric tower and The length of the line segment It can be calculated using the Euclidean distance formula in three-dimensional space:

[0041]

[0042] Only when The distance is calculated and added to the inspection path of the drone.

[0043] The total route length L of the drone inspection is the sum of the lengths of all connected route segments:

[0044]

[0045] like Figure 2 As shown in the figure, since UAVs cannot communicate directly with satellites but instead communicate through ground satellite terminals, the space-ground integrated network architecture needs to consider the communication relay relationship between UAVs and ground satellite terminals.

[0046] The drones communicate through a flying ad hoc network, and each drone acts as a mobile node. Suppose the drone swarm has UAVs, each UAV’s position is , .

[0047] Drones and drones Communication distance between Calculated by the following formula:

[0048]

[0049] if , is the communication radius of the drone, then the two drones can communicate directly. Otherwise, they need to be relayed by other drones. For drones and drones The distance .

[0050] The ground satellite terminal is responsible for communicating with the satellite. Assume that the position of the ground satellite terminal is The communication distance between the UAV and the ground satellite terminal is:

[0051]

[0052] If the distance between the drone and the satellite terminal (communication radius of the satellite terminal), direct communication is possible; otherwise, relay is performed through the flying ad hoc network.

[0053] In order to ensure the stability and efficiency of communication, the drone swarm maintains communication with the ground satellite terminal through the flying ad hoc network. The ground satellite terminal is responsible for communicating with the satellite, and the drone transmits data to the terminal through the optimal path. When selecting the optimal path, it is necessary to consider the balance between communication delay and energy consumption.

[0054] The energy consumption of each drone mainly depends on its communication and flight energy consumption. Assuming that each drone The remaining energy is ,When selecting the communication path, the energy consumption of the entire drone swarm needs to be balanced.

[0055] The total energy consumption of a drone should include two parts:

[0056] 1. Flight energy consumption : The energy consumed by the UAV during the inspection process in the tth time period due to flight;

[0057] 2. Communication energy consumption : Energy consumption when a UAV communicates with other UAVs or ground satellite terminals.

[0058] Communication energy consumption It represents the energy consumption of a UAV when communicating with other UAVs or ground satellite terminals through a flying ad hoc network. The communication energy consumption of a UAV can be expressed as:

[0059]

[0060] in:

[0061] It is The communication power of the UAV in the tth time period;

[0062] It is The communication time of the UAV in the tth time period is the communication time with other nodes (other UAVs or ground satellite terminals) during the mission.

[0063] Comprehensive flight energy consumption and communication energy consumption, the tth time period The total energy consumption model of a UAV is:

[0064] .

[0065] The drone status can be obtained through the following data:

[0066] State Space is the input of the algorithm, describing the current system environment information and representing the state of the drone at each moment. In the routing problem of drone swarms, the state space should include information related to the location, energy, and communication status of the drone. The status of a drone should include:

[0067] Current drone position: the 3D coordinates of the drone at the current moment , They are the coordinates of the three axes at the current moment t.

[0068] Future location: The drone’s expected location at the next few moments. ,in is the future time step. This information can be used to predict future communication link conditions and delay fluctuations and optimize routing selection.

[0069] Surplus Energy: Drones The remaining energy , used to control energy consumption and task execution time.

[0070] Current Communication Latency: Drones Current communication delay with ground satellite terminals .

[0071] Neighboring node status: including the current communication status of other nearby drones and ground satellite terminals , as well as the relative locations of these nodes and the quality of communication links.

[0072] Therefore, the The state of a drone can be expressed as:

[0073]

[0074] The state space of the entire drone swarm is the set of all drone states:

[0075]

[0076] Action Space Determines the routing of drones in the communication network. In communication routing, each drone needs to decide which node to send data to to ensure reliable data transmission and minimize communication delay and energy consumption.

[0077] Select communication node: The drone can choose to establish communication with other drones or ground satellite terminals within its communication range. Represented as the selected communication target node:

[0078] .

[0079] In a specific embodiment of the present invention, the reward function is:

[0080]

[0081] is the reward function value of the entire drone swarm in the tth time period; is the sum of the communication path delays of the entire drone swarm in the tth time period; is the energy imbalance of the entire drone swarm in the tth time period; is the sum of the communication delay jitter of the entire drone group in the tth time period; is the communication energy consumption of the entire drone group in the tth time period, = , For the The communication energy consumption of a UAV in the tth time period; , , and They are delay weight coefficient, energy imbalance weight coefficient, delay jitter weight coefficient and communication energy consumption weight coefficient respectively.

[0082] The entire flight time is divided into multiple time periods of equal length. The time periods are cut into very small pieces, so the communication action within the time period is stable and does not need to be switched. For example, the time period is 0.1 seconds or 0.01 seconds. Of course, the time period can also be set according to the communication frequency.

[0083] By associating the reward function with communication energy consumption and communication delay, and the reward function is negatively correlated with communication energy consumption and communication delay, when optimizing the target, try to choose a reward function with a large value for optimization, that is, optimize the communication strategy in the direction of low communication energy consumption and communication delay. In addition, when considering communication energy consumption and communication delay, related factors such as energy consumption imbalance and delay jitter are also considered, so that the effective endurance working time and communication stability of the entire system are guaranteed.

[0084] In a specific embodiment of the present invention, the sum of the communication path delays Calculated according to the following formula: ; .

[0085] is the tth time period The path delay between the UAV and the ground satellite terminal, is the tth time period The delay of the i-th communication transmission path of the UAV; is the total number of drones in the drone swarm; For the The total number of segments of the communication transmission path of the UAV, such as The UAV needs to pass through two intermediate UAVs to transmit data to the ground satellite terminal. The total number of segments of the communication transmission path of the UAV is 3.

[0086] In a specific embodiment of the present invention, the communication delay jitter sum Calculated according to the following formula: ; ; is the tth time period Delay jitter of the drone; is the t-1th time period The path delay between the drone and the ground satellite terminal. Delay jitter refers to the fluctuation range between the transmission delays of adjacent data packets, that is, the degree of change in the transmission time of each data packet. This parameter can reflect the fluctuation of the delay of the drone in the process of communicating with other nodes, and whether the fluctuation of communication delay is drastic, so as to measure the stability of communication delay and avoid unstable communication caused by drastic fluctuation of communication delay.

[0087] In a specific embodiment of the present invention, the energy imbalance It can be calculated by the following formula: .

[0088] is the energy value of the drone with the largest remaining energy in the drone swarm after the end of the tth time period, is the energy value of the drone with the smallest remaining energy in the drone group after the end of the tth time period. When the drone group is working, if there is a large energy imbalance, that is, some drones have more energy and some drones have less energy, the effective working time of the entire drone group will be limited. Therefore, when selecting communication actions, it is necessary to consider whether the energy of the drone group is balanced. When selecting communication actions, it is appropriate to consider reducing the power consumption of drones with less energy, and it is appropriate to ignore this behavior to increase the power consumption of drones with more energy. The degree of appropriate consideration is determined by the weight This decision makes it possible to optimize beyond simply considering total power consumption.

[0089] In a specific embodiment of the present invention, the optimization function and optimization strategy of this method use the PPO algorithm. The PPO algorithm first interacts with the environment through the current strategy to collect the state, action, and reward data of the drone. ,action and corresponding rewards Construct trajectory data. The trajectory includes data of multiple time steps ,in The total number of time periods for a task. The collected data is used for subsequent strategy optimization. Specifically, in the communication routing scenario, the system collects the following information: .here Indicates The drone The state of a time period, is its communication action, It is in the The rewards for each time period are accumulated for all drones. .

[0090] The strategy optimization function is ,

[0091] ;

[0092] are policy parameters. In deep reinforcement learning, the policy is usually parameterized by a neural network. The weights and biases of the neural network are the policy parameters. These parameters determine the strategy In PPO, the goal of policy optimization is to adjust the parameter θ so that the drone swarm is more likely to choose a high-return action under a specific state. By collecting the interaction data between the drone swarm and the environment, the algorithm will use optimization methods such as gradient descent to gradually update the policy parameter θ to improve the performance of the policy. The function value is larger. is the state of the drone swarm in the tth time period; represents the expectation of all future returns based on the current strategy starting from the tth time period; To find the minimum function; is a range-limiting function; The clipping threshold for policy updates; is the importance sampling ratio; is the advantage function of the drone swarm in the tth time period, is the communication action of the drone group in the tth time period.

[0093] .

[0094] PPO by Importance Sampling Ratio To adjust the update amplitude of the strategy and ensure the stability of the strategy update. The importance sampling ratio is defined as:

[0095]

[0096] in It's an old strategy. is the new strategy. This ratio indicates that the new and old strategies are in the same state Select the same action The probability ratio.

[0097] In a specific embodiment of the present invention, the advantage function The specific calculation formula is as follows:

[0098] ;

[0099] is the reward weight factor; The status is The expected total return value at is the number of time periods in the entire communication task.

[0100] Usually refers to the current reward expectation. Specifically, Indicates that starting from the tth time period, based on the current strategy The expectation of all future returns. This is one of the important indicators used by the algorithm to optimize the strategy. In the application of the PPO algorithm, can be considered as the current expected cumulative return; by calculating , it can be estimated that the reward that can be obtained in the future after taking actions in the current state.

[0101] ;

[0102] Indicated in strategy Next expectations. is the discount factor, which indicates the weight of future rewards; is the immediate reward in the t'th time period. It can also be regarded as a state value function In the PPO algorithm, the value function Indicates that it is in the state Next, based on the current strategy , starting from the tth time period, the expected value of all possible cumulative rewards in the future, that is:

[0103] ;

[0104] Indicates in status and strategies Status expectations.

[0105] In the communication routing task of drone swarm, It reflects the expected future rewards after selecting a communication node or routing decision. ,The PPO algorithm can optimize the UAV’s communication routing strategy and balance the ,communication delay, energy consumption and delay jitter.

[0106] State Value Function Focus on the expected cumulative reward in a specific state. It represents the expected cumulative reward from a certain state. At the beginning, the total reward that can be obtained by executing the strategy in the future. Reward expectation It refers to the cumulative reward of the entire task, not just the reward in a specific state. It measures the overall performance of the entire strategy and is the goal of strategy optimization. Advantage function Measures the "goodness" of choosing a particular action in a state relative to all possible actions in that state. It reflects whether choosing a certain action performs better than the average policy in the state.

[0107] The method of updating and iterating the communication actions of the drone swarm according to the strategy optimization function to obtain the optimal communication action routing strategy of the drone swarm specifically includes steps A1, A2 and A3.

[0108] A1, based on the objective function The PPO algorithm optimizes the strategy through the gradient descent method and updates the strategy parameters Specifically, through the gradient To adjust the parameters of the strategy so that the strategy gradually tends to the optimal. Represents the objective function About strategy parameters It is a vector that contains the rate of change of the objective function for each parameter, indicating how to adjust the policy parameters To increase the value of the objective function. The gradient direction represents the objective function The fastest direction of increase. During the optimization process, the algorithm calculates To update the parameters, the objective function Increase, thereby optimizing the strategy.

[0109] A2, after a strategy update, the PPO algorithm repeats the above steps. After each iteration, the strategy It is continuously improved until it converges to the optimal strategy. Finally, the drone swarm obtains the optimal communication routing strategy in a complex communication environment, which can achieve the optimal balance between communication delay, energy consumption and delay jitter. The principle is as follows Figure 3 , Figure 4 shown.

[0110] A3, through multiple iterations of optimization, the PPO algorithm generates a set of optimal communication routing strategies. , drones According to the current status And the optimal strategy learned by PPO To select the best communication node to ensure the efficiency and stability of data transmission.

[0111] In a specific embodiment of the present invention, when the distance between the drone and other adjacent drones is less than a first preset communication distance, the drone can choose to transmit data to other adjacent drones. When the distance between the drone and the ground satellite terminal is less than a second preset communication distance, the drone can choose to transmit data to the ground satellite terminal. , thereby limiting the selection of communication actions and avoiding unstable signal data transmission due to excessive communication distance.

[0112] The present invention also proposes an electronic device, including a processor and a memory, wherein the memory is used to store program code and transmit the program code to the processor; the processor is used to execute the unmanned aerial vehicle swarm integrated space-ground communication routing method according to the instructions in the program code.

[0113] The present invention also proposes a ground-ground integrated communication routing system for a swarm of unmanned aerial vehicles, comprising a state acquisition module, a reward function establishment module, a strategy optimization function establishment module, a strategy update optimization module, and an instruction sending module.

[0114] The status acquisition module is used to obtain the status of the drone fleet.

[0115] The reward function establishment module is used to establish a reward function regarding the state of the drone group and the communication action, wherein the drone transmits data to other nearby drones or ground satellite terminals according to the communication action.

[0116] The strategy optimization function establishment module is used to establish the strategy optimization function according to the reward function.

[0117] The strategy update optimization module is used to update and iterate the communication actions of the drone swarm according to the strategy optimization function to obtain the optimal communication action routing strategy of the drone swarm.

[0118] The command sending module is used to send communication commands to the drone fleet.

[0119] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for integrating ground-ground communication routing of drone swarms, characterized in that: The steps include: Get the status of the drone fleet; Establishing a reward function for the state of a drone group and a communication action, wherein the drone transmits data to other neighboring drones or a ground satellite terminal according to the communication action, wherein the ground satellite terminal is used to communicate with a satellite, wherein the reward function is associated with communication energy consumption and communication delay, and the reward function is negatively correlated with the communication energy consumption and the communication delay; According to the reward function, establish a strategy optimization function; Update and iterate the communication actions of the drone swarm according to the strategy optimization function to obtain the optimal communication action routing strategy of the drone swarm; Sending a communication instruction to the drone fleet, wherein the communication instruction includes an optimal communication action routing strategy; The reward function is: ； is the reward function value of the entire drone swarm in the tth time period; is the sum of the communication path delays of the entire drone swarm in the tth time period; is the energy imbalance of the entire drone swarm in the tth time period; is the sum of the communication delay jitter of the entire drone group in the tth time period; is the communication energy consumption of the entire drone group in the tth time period; , , and They are delay weight coefficient, energy imbalance weight coefficient, delay jitter weight coefficient and communication energy consumption weight coefficient respectively; The energy imbalance It can be calculated by the following formula: ； is the energy value of the drone with the largest remaining energy in the drone swarm after the end of the tth time period, It is the energy value of the drone with the smallest remaining energy in the drone swarm after the end of the tth time period.

2. The method for integrated space-ground communication routing of drone swarms according to claim 1, characterized in that: The sum of the communication path delays Calculated according to the following formula: ； ； is the tth time period The path delay between the UAV and the ground satellite terminal, is the tth time period The delay of the i-th communication transmission path of the UAV; is the total number of drones in the drone swarm; For the The total number of segments of the communication transmission path of the UAV.

3. The method for integrated space-ground communication routing of drone swarms according to claim 2, characterized in that: The communication delay jitter sum Calculated according to the following formula: ； ； is the tth time period Delay jitter of the drone; is the t-1th time period The path delay between the UAV and the ground satellite terminal.

4. The method for integrated space-ground communication routing of drone swarms according to claim 1, characterized in that: The strategy optimization function is , ； is the strategy parameter, is the communication action of the drone group in the tth time period; is the state of the drone swarm in the tth time period; represents the expectation of all future returns based on the current strategy starting from the tth time period; To find the minimum function; is a range-limiting function; The clipping threshold for policy updates; is the importance sampling ratio; is the advantage function of the drone swarm in the tth time period.

5. The method for integrated space-ground communication routing of drone swarms according to claim 4, characterized in that: The advantage function The specific calculation formula is as follows: ； is the reward weight factor; The status is The expected total return value at is the number of time periods in the entire communication task.

6. The method for integrated space-ground communication routing of drone swarms according to claim 1, characterized in that: When the distance between the drone and other neighboring drones is less than a first preset communication distance, the drone can choose to transmit data to other neighboring drones.

7. The method for integrated space-ground communication routing of drone swarms according to claim 1, characterized in that: When the distance between the UAV and the ground satellite terminal is less than a second preset communication distance, the UAV can choose to transmit data to the ground satellite terminal.

8. An electronic device, characterized in that: including a processor and a memory, The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the drone swarm ground-to-sky integrated communication routing method as described in any one of claims 1-7 according to the instructions in the program code.

9. A drone swarm integrated ground-ground communication routing system, characterized in that: include: Status acquisition module, used to obtain the status of the drone fleet; A reward function establishment module, used to establish a reward function for the state of the drone group and the communication action, wherein the drone transmits data to other adjacent drones or ground satellite terminals according to the communication action, and the reward function is associated with the communication energy consumption and the communication delay, and the reward function is negatively correlated with the communication energy consumption and the communication delay; A strategy optimization function establishment module is used to establish a strategy optimization function based on the reward function; The strategy update optimization module is used to update and iterate the communication actions of the drone swarm according to the strategy optimization function to obtain the optimal communication action routing strategy of the drone swarm; A command sending module is used to send communication commands to the drone fleet; The reward function is: ； is the reward function value of the entire drone swarm in the tth time period; is the sum of the communication path delays of the entire drone swarm in the tth time period; is the energy imbalance of the entire drone swarm in the tth time period; is the sum of the communication delay jitter of the entire drone group in the tth time period; is the communication energy consumption of the entire drone group in the tth time period; , , and They are delay weight coefficient, energy imbalance weight coefficient, delay jitter weight coefficient and communication energy consumption weight coefficient respectively; The energy imbalance It can be calculated by the following formula: ； is the energy value of the drone with the largest remaining energy in the drone swarm after the end of the tth time period, It is the energy value of the drone with the smallest remaining energy in the drone swarm after the end of the tth time period.

Images