Unmanned aerial vehicle communication network connectivity recovery method and system based on MADDPG and APF

Abstract

The invention relates to the technical field of unmanned aerial vehicle communication networks, in particular to an unmanned aerial vehicle communication network connectivity recovery method and system based on MADDPG and APF, and the method comprises the steps: building a network connectivity recovery task model; designing a network connectivity recovery mechanism; defining an action space, a state space and a reward function; designing a network connectivity recovery model based on the MADDPG and the APF, and training the network connectivity recovery model; and realizing network connectivity recovery by using the trained network connectivity recovery model. According to the invention, a network connectivity recovery model based on MADDPG and APF design is designed based on a network connectivity recovery mechanism, and connectivity between disconnected clusters is recovered at a faster speed and a shorter total moving distance.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle communication networks, and in particular to a method and system for restoring connectivity of unmanned aerial vehicle communication networks based on MADDPG and APF. Background Art

[0002] The UAV communication network is a network system composed of multiple UAVs. UAVs usually dynamically establish and maintain communication links and do not rely on fixed communication infrastructure. Network connectivity ensures real-time information exchange and data transmission between UAVs to promote collaboration and information sharing between UAVs, so as to perform tasks more effectively and accurately. However, during flight, UAVs face many threats that may cause damage, such as strong electromagnetic interference, network attacks, severe weather conditions, and air defense weapon systems. However, if a large number of UAVs are damaged, the UAV cluster will split into multiple disconnected clusters. The distance between clusters is so far that strategies such as replacing damaged UAVs with remaining UAVs and enhancing the communication radius of UAVs may not be able to restore connectivity. In addition, due to the limited communication distance, each cluster cannot determine whether some other UAVs are in working or damaged state, nor can it determine the location of some other existing UAVs and the exact number and location of damaged UAVs. Therefore, restoring connectivity between multiple disconnected clusters faces great challenges. Summary of the invention

[0003] The present invention aims to solve the problem that the entire cluster of drones is split into multiple disconnected clusters due to large-scale damage to drones. A method and system for restoring the connectivity of drone communication networks based on MADDPG and APF are proposed to restore the connectivity between disconnected clusters at a faster speed and a shorter total moving distance.

[0004] To achieve the above purpose, the technical solution adopted is:

[0005] A UAV communication network connectivity recovery method based on MADDPG and APF, comprising:

[0006] Establish a network connectivity restoration task model;

[0007] Design network connectivity restoration mechanisms;

[0008] Define action space, state space, and reward function;

[0009] Design and train a network connectivity restoration model based on MADDPG and APF;

[0010] Use the trained network connectivity restoration model to achieve network connectivity restoration.

[0011] According to the UAV communication network connectivity restoration method based on MADDPG and APF of the present invention, further, establishing a network connectivity restoration task model includes:

[0012] The goal of connectivity recovery is to find new locations for the remaining drones in the drone cluster after removing the damaged drone, so that each cluster can be restored to connection as soon as possible, and the total movement distance of the remaining drones is as short as possible; to achieve the connectivity recovery goal, it is necessary to minimize the maximum value of the movement time of all remaining drones and the total movement distance of the remaining drones.

[0013] According to the UAV communication network connectivity recovery method based on MADDPG and APF of the present invention, further, the network connectivity recovery mechanism includes UAV damage identification, and the specific process is as follows:

[0014] First, each remaining drone processes the received message. Received u j location information; if Received u j Status information Indicates u j Normal operation; Indicates u j Damaged; otherwise Indicates u j The state of is unknown; at time t, establish The remaining drone set and the uncertain drone set;

[0015] Then, each remaining UAV constructs an undirected graph G based on its position information and state information i (U i ,E i );

[0016] Finally, calculate G i (U i ,E i )’s Laplace matrix L i The eigenvalues ​​of L i The feature values ​​of are used to draw conclusions about whether there are other clusters.

[0017] According to the UAV communication network connectivity recovery method based on MADDPG and APF of the present invention, further, the network connectivity recovery mechanism includes the UAV position relocation based on the leader-follower mechanism, specifically including:

[0018] (1) Restoring connectivity

[0019] For each cluster, the drone closest to the cluster center is selected as the leader and the other drones are followers. Each leader moves at a maximum speed v max Flying towards the center of the cluster, each follower selects a superior drone to maintain connection with, so the entire cluster is connected;

[0020] (2) Accelerate the restoration of connectivity

[0021] First, from the perspective of the uncertain drones, each uncertain drone is assigned to its closest follower; if a follower has no uncertain drones to explore, but its neighbors do, then it regards all the uncertain drones that its neighbors are going to explore as its own uncertain drones to explore, and cooperates with its neighbors;

[0022] Next, to find all leaders, all uncertain drones are considered as virtual leaders moving toward the center of the cluster. quilt Successful exploration means that at a certain time t, D(i,j)≤R is satisfied, where By exploring the virtual leader, it is determined whether the uncertain UAV corresponding to the virtual leader is the leader; in addition, at the exploration time t, the follower You may encounter unsure drones that work well at this time Then Can be used with All the uncertain drones in the cluster that are running well establish communication, and it is possible to know whether these uncertain drones are leaders, without having to explore their corresponding virtual leaders;

[0023] (3) Reduce unnecessary movement of remaining drones

[0024] Each follower closest to the uncertain UAV and its neighbors should actively explore the uncertain UAV, while other followers do not need to.

[0025] According to the UAV communication network connectivity recovery method based on MADDPG and APF of the present invention, further, judging whether the uncertain UAV corresponding to the virtual leader is the leader includes:

[0026] ① When Successful exploration at time t If it can be located at If the drone establishes communication, the drone is not sure is well run, it is the leader, Can Communicate with all followers in the cluster;

[0027] ② When Successful exploration at time t , but it cannot be compared with If the drone establishes communication, the drone is not sure Instead of being a leader, it can be damaged or become a follower;

[0028] In the above process, if Discover Is running well, need to The cluster and The clusters they are in are merged, the leader stops moving when it reaches the center of the cluster, and each follower stops moving when the conditions for restoring connectivity are met.

[0029] According to the UAV communication network connectivity recovery method based on MADDPG and APF of the present invention, the action space is further defined as:

[0030] If the follower If the communication with the upper drone is about to be lost, or if the drone is about to collide with the surrounding drones, Use APF for path planning; otherwise, Gain speed with MADDPG.

[0031] According to the UAV communication network connectivity recovery method based on MADDPG and APF of the present invention, the state space is further defined as: for local observations, The drone observed was The remaining drones and uncertain drones except the leader within the communication range; for global observation, All uncertain drones and all remaining drones were observed.

[0032] According to the UAV communication network connectivity restoration method based on MADDPG and APF of the present invention, further, the reward function is defined as the reward for completing the task of restoring connectivity, the reward and encouragement for the remaining UAVs involved in the local observation to successfully explore an uncertain UAV The sum of rewards for exploring uncertain drones.

[0033] According to the UAV communication network connectivity recovery method based on MADDPG and APF of the present invention, further, the training process of designing a network connectivity recovery model based on MADDPG and APF is as follows:

[0034] Randomly select multiple drones and put them in a damaged state;

[0035] Start UAV damage identification;

[0036] If the network is not connected, obtain the initial local observations and initial global observations of all remaining drones;

[0037] For each damaged drone or leader that has reached the cluster center, the action is the zero vector;

[0038] For each leader that has not reached the cluster center, the movement direction points to the cluster center, and the size is the maximum speed value;

[0039] For each follower, if the follower is about to lose communication with the parent drone or is about to collide with the surrounding drones, use APF to obtain the speed; otherwise, use MADDPG to obtain the speed;

[0040] Execute actions and obtain rewards, obtain new local and global observations;

[0041] Store the last local observation, the last global observation, the velocity, the new local observation, and the new global observation in the experience replay buffer;

[0042] Randomly sample and update the parameters of the policy network, value network, target policy network, and target value network from the experience replay buffer;

[0043] If the remaining drones determine that connectivity is restored, the round ends.

[0044] Furthermore, the present invention also provides a UAV communication network connectivity recovery system based on MADDPG and APF, comprising:

[0045] A task model building module is used to build a network connectivity restoration task model;

[0046] The recovery mechanism module is used to design the network connectivity recovery mechanism;

[0047] Space definition module, used to define action space, state space and reward function;

[0048] Model training module, used to design and train network connectivity restoration models based on MADDPG and APF;

[0049] The connectivity restoration module is used to restore network connectivity using the trained network connectivity restoration model.

[0050] The beneficial effects achieved by adopting the above technical solution are:

[0051] 1. The present invention designs a complete network connectivity recovery mechanism, including drone damage identification based on location information and status information, and drone position relocation based on the leader-follower mechanism. On the one hand, it can accurately identify whether the entire cluster is split into multiple unconnected clusters. On the other hand, it accelerates the restoration of connectivity by actively approaching uncertain drones through the followers of each cluster. Furthermore, some followers are kept stationary and maintain a safe connection with their superiors, thereby reducing the total movement distance of the drone.

[0052] 2. Design a network connectivity recovery model based on MADDPG and APF based on network connectivity recovery mechanism. This model can restore the connectivity between disconnected clusters with faster speed and shorter total moving distance. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings of the embodiments of the present invention, wherein the drawings are only used to illustrate some embodiments of the present invention, but not to limit all embodiments of the present invention thereto.

[0054] Figure 1 It is a flowchart of a method for restoring connectivity of a UAV communication network based on MADDPG and APF according to an embodiment of the present invention;

[0055] Figure 2 is an example diagram of an entire cluster being split into multiple disconnected clusters according to an embodiment of the present invention;

[0056] Figure 3 is an example diagram of realizing connectivity restoration according to an embodiment of the present invention;

[0057] Figure 4 In the embodiment of the present invention, in the process of restoring connectivity, the follower selects MADDPG or APF to obtain a speed diagram;

[0058] Figure 5 is a schematic diagram of the average recovery time required for restoring connectivity of different numbers of damaged drones according to an embodiment of the present invention;

[0059] Figure 6 Schematic diagram of the average moving distance of all remaining drones under different numbers of damaged drones according to an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings of specific embodiments of the present invention to clearly and completely describe the exemplary scheme of the embodiment of the present invention. Unless otherwise defined, the technical terms or scientific terms used in the present invention should be the common meanings understood by people with ordinary skills in the field.

[0061] like Figure 1As shown, this embodiment discloses a method for restoring connectivity of a UAV communication network based on MADDPG and APF, comprising the following steps:

[0062] Step S101: Establish a network connectivity restoration task model.

[0063] A cluster of m drones is represented by U = {u1,u2,...,u m Assume that at time t1, d(d∈{2,3,..,m-2}) drones are damaged, and the entire cluster is split into multiple disconnected clusters, such as Figure 2 As shown in Figure 1, the damage to the gray drones causes the black drones to form three disconnected clusters. The remaining drones are recorded as lie in The connectivity restoration goal is to find new positions P′={p′1,p′2,...,p′} for the remaining UAVs. m-d}, so that each cluster is connected as quickly as possible, and the total movement distance of the remaining drones is as short as possible. Actual recovery means that all remaining drones have been reorganized into a connected cluster, but each drone is not sure that there are no other remaining drones outside the drone cluster. Determined recovery means that all remaining drones have been reorganized into a connected cluster, and each drone is sure that there are no other remaining drones outside the drone cluster. Only when all remaining drones achieve deterministic recovery can connectivity be truly restored. To achieve this goal, it is necessary to minimize the maximum value of the movement time of all remaining drones and the total movement distance of the remaining drones, that is:

[0064]

[0065] Where T i express From Location Move to position p′ i The time spent, E r ={e ij |||p i,t -p j,t ||≤R,i≠j,i,j∈{1,2,...,md}}, R is the communication radius of the drone. This is an NP-hard problem, so the problem is divided into three aspects: restoring connectivity, reducing the time to restore connectivity, and reducing unnecessary movements of the remaining drones.

[0066] Step S102: Design a network connectivity recovery mechanism.

[0067] The connectivity recovery mechanism includes drone damage identification and drone position relocation.

[0068] Drone damage identification is to determine whether the entire cluster is split into multiple unconnected clusters. The specific process is as follows:

[0069] First, each remaining drone processes the received message. Assume that a broadcast is started every Δt, and a new round of broadcast occurs at time t. If it receives u j The location information p j,t ,but otherwise u j Status information j,t Possible values ​​are 0 and 1, s j,t =1 means u j Normal operation;s j,t =0 means u j damaged. If Received information about u j Status information otherwise (indicates u j The state of is unknown). At time t, The uncertain drone set is denoted as At time t, The remaining drone set is recorded as The set of uncertain drones for each cluster is different from the set of remaining drones.

[0070] Then, each remaining drone is based on and Construct an undirected graph G i (U i ,E i ). The initial U i and E i Is empty. If In U i Add u j .if And||p i,t -p j,t ||≤R, then edge e ij Add to E i middle.

[0071] Finally, calculate G i (U i ,E i )’s Laplace matrix L i The eigenvalues ​​of L i The eigenvalues ​​of L i =D i -A i , where A i and D i G i (Ui ,E i )’s adjacency matrix and degree matrix. If L i There is only one zero eigenvalue, then the graph G i (U i ,E i ) are connected, Know that there are no other clusters. Otherwise, graph G i (U i ,E i ) is not connected, Know that other clusters may exist.

[0072] Once it is determined that the entire drone cluster may have split into multiple disconnected clusters, all remaining drones need to fly to appropriate locations to restore connectivity. This case designs a drone position relocation method based on the leader-follower mechanism, the specific contents are as follows:

[0073] (1) Restoring connectivity

[0074] For each cluster, the drone closest to the cluster center is selected as the leader and the other drones as followers, e.g. Figure 3 As shown in the figure, the drones marked as A1, B1, and C1 are leaders, and the other drones are followers. Each leader moves at a maximum speed v max Each follower selects a suitable drone to maintain connection to ensure that the cluster does not split again and avoid collisions. First, each cluster is layered as follows: Neighbors The minimum number of hops to the leader is smaller, then for of the upper drone; if The minimum number of hops to the leader is The same, then The same level drone. The closest drone is selected as the superior from the upper drones and the same-layer drones that are closer to the leader. If every follower is connected to its superior, then the entire cluster is connected.

[0075] (2) Accelerate the restoration of connectivity

[0076] Determine the uncertain drones to be explored for each follower and design an exploration mechanism to restore connectivity faster. First, determine the cooperative drones and uncertain drones to be explored for each follower; each follower cooperates with other followers within the communication range. The method for determining the uncertain drones to be explored by each follower is shown in Algorithm 1. Specifically, from the perspective of the uncertain drone, each uncertain drone is assigned to the follower closest to it. If a follower has no uncertain drones to explore, but its neighbors do, then it regards all the uncertain drones to be explored by its neighbors as its own uncertain drones to be explored and cooperates with its neighbors. The uncertain drones to be explored by each follower need to be updated only when it is found that the uncertain drone is running. Otherwise, the uncertain drones to be explored by each follower remain unchanged. When determining the uncertain drones to be explored by each follower, each follower needs to know the locations of all the remaining drones in the cluster. Otherwise, each follower only needs to regularly update and maintain the location information of the remaining drones within its communication range to complete the exploration task. This local communication mechanism can reduce unnecessary global communication, reduce communication overhead, and improve system performance.

[0077] Next, we will introduce how followers explore uncertain drones. The true state of an uncertain drone is either running well or damaged. If an uncertain drone is found not to be running well, it must be destroyed. No matter how many clusters the uncertain drones form, as long as all other leaders are found, all the uncertain drones that are running well can be known, thereby establishing communication with all the uncertain drones that are running well, and knowing which uncertain drones are damaged. Therefore, the key is to find all leaders. Since the state of the uncertain drones is unknown, each uncertain drone may be a leader. In addition, the movement strategy of the leader is fixed. By finding all leaders, the exploration of all uncertain drones is completed. In order to find all leaders, all uncertain drones are regarded as virtual leaders moving towards the center of the cluster, expressed as quilt Successful exploration means that at a certain time t, D(i,j)≤R is satisfied, where By exploring the virtual leader, we determine whether the uncertain drone corresponding to the virtual leader is the leader, and thus find all other leaders. The specific analysis is as follows:

[0078] ① When Successful exploration at time t If it can be located at If the drone establishes communication, the drone is not sure It is well run and it is the leader. Able to All followers that actually exist in the cluster communicate with each other. These followers cannot be leaders, so there is no need to explore their corresponding virtual leaders.

[0079] ② When Successful exploration at time t , but it cannot be compared with If the drone establishes communication, the drone is not sure Instead of being a leader, it can be damaged or become a follower.

[0080] In addition, at exploration time t, the follower You may encounter unsure drones that work well at this time Then Can be used with By establishing communication with all the uncertain drones that are running well in the cluster, it is possible to know whether these uncertain drones are leaders, without having to explore their corresponding virtual leaders.

[0081] In the above process, if Discover It works well because and belong to two different clusters, so the two clusters need to be merged. The new leader is the drone closest to the center among the leaders from the merged cluster. The two clusters can exchange the information they know. and Broadcast separately and Received or Drones renew as follows:

[0082]

[0083] After exploring all virtual leaders, if there are still but Update to 0. After merging, each drone in the new cluster can also obtain and update the location information of other drones in the new cluster. The leader stops moving when it reaches the center of the cluster, and each follower stops moving when the following conditions are met.

[0084]

[0085] (3) Reduce unnecessary movement of remaining drones

[0086] Not all followers need to actively explore uncertain drones. Some followers may have maximized the speed of connectivity recovery. Therefore, a method is designed to determine whether a follower needs to actively explore uncertain drones. Specifically, the follower closest to an uncertain drone and its neighbors should actively explore the uncertain drone, while other followers do not need to. Figure 3 For example, for cluster C, the state of drone B1 is uncertain, and follower C2 is closest to it, so C2 needs to actively explore B1. In this case, follower A3 can remain stationary because it maintains a secure connection with its superior, without the risk of connectivity interruption, without the need to avoid collisions with surrounding drones, and without the need to explore uncertain drones.

[0087] Step S103: define the action space, state space and reward function.

[0088] (1) Define the action space

[0089] The action of each drone is speed. If the follower About to meet with superior Loss of communication, or imminent collision with surrounding drones, APF will be used for path planning. Specifically, Will As the target, and consider your distance no greater than r rep The drone is considered as an obstacle, and the speed is obtained using APF As shown below:

[0090]

[0091] Among them, r rep yes The minimum distance to the surrounding drones, c att is the coefficient of the gravitational term, c rep is the coefficient of the repulsive term, and p(i,j)=p i,t -p j,t .otherwise, Will gain speed through MADDPG During the training phase, you need to Add Gaussian noise. In short, The speed can be expressed as:

[0092]

[0093] Among them, r att yes With his superior The maximum value of the distance.

[0094] from Figure 4 It can be seen that in the process of restoring connectivity, each follower will choose MADDPG or APF to obtain speed according to its position relationship with other UAVs.

[0095] (2) Define the state space

[0096] Followers need to cooperate with each other to explore the uncertain UAV. Therefore, both local and global observations are related to the relative positions of the uncertain UAV and the remaining UAVs.

[0097] For local observations, The remaining drones observed were All other drones except the leader within the communication range, Determine the set of observed uncertain drones according to Algorithm 1. For global observations, Consider all known uncertain drones and all remaining drones.

[0098]

[0099] (3) Define the reward function

[0100] At time t, for For example, the reward for completing the task of restoring connectivity is Wherein, c1 is a positive number. The reward for successfully exploring an uncertain drone by the remaining drones involved in the local observation of Among them, c2 and c3 are both positive numbers. Encourage Exploring the rewards of uncertain drones for In summary, the reward function is

[0101] When a drone is a leader or damaged, its local observation, global observation, action, and reward are set to zero vector or zero, which does not affect the drone learning policy.

[0102] Step S104: design a network connectivity restoration model based on MADDPG and APF and perform training.

[0103] A complete training process is for the same initial topology. The trained network can cope with a variety of drone damage situations. The pseudo code of the model in the training phase is shown in Algorithm 2.

[0104]

[0105]

[0106] Step S105: Use the trained network connectivity restoration model to restore network connectivity.

[0107] Initially, drones are randomly distributed in the working area and the communication network topology is connected. Some drones are damaged, causing the entire cluster to split into multiple disconnected clusters. The remaining drones obtain appropriate actions based on local observations and the trained policy network until a new connected cluster is obtained.

[0108] Corresponding to the above method, this embodiment also proposes a UAV communication network connectivity recovery system based on MADDPG and APF, including:

[0109] The task model building module is used to establish the network connectivity restoration task model.

[0110] The recovery mechanism module is used to design the network connectivity recovery mechanism.

[0111] The space definition module is used to define the action space, state space and reward function.

[0112] The model training module is used to design and train a network connectivity restoration model based on MADDPG and APF.

[0113] The connectivity restoration module is used to restore network connectivity using the trained network connectivity restoration model.

[0114] In order to verify the feasibility and effectiveness of the method, the present invention is further described in detail below in conjunction with a simulation example.

[0115] The simulation environment is constructed using Python to verify the effectiveness of the method. The working area of ​​the simulation environment is a two-dimensional space of 1000×1000. Initially, 20 drones are randomly distributed in the middle area of ​​600×600. The maximum speed of the drone is 1, and the communication radius of the drone is 200. The distance between the two drones is not less than the safety distance d min , set to 1. The relevant parameters of APF are set as follows: rep =0.5, r rep =10, c att =1.5, r att =190. The relevant parameters of the reward function are set as follows: c1=200, c2=500, c3=5. Both the policy network and the value network have three fully connected layers. The learning rates of the policy network and the value network are 0.0005 and 0.001, respectively. The discount factor γ is 0.99 and the update factor τ is 0.01. During the training phase, the total number of rounds is 50,000, and the maximum number of steps per round is 300. The capacity of each experience replay buffer is 10,000, and the batch size J is equal to 512.

[0116] The proposed connectivity restoration method is evaluated in terms of the time taken to restore connectivity and the total distance traveled by all remaining drones to restore connectivity. Tests are performed with 2, 3, …, 18 damaged drones. For a given number of damaged drones, the number of tests is the total number of damaged cases and the maximum of 1000, and the average time and the average distance traveled by all remaining drones are obtained.

[0117] The contrast method is that all remaining drones move directly to the center of the cluster, denoted by CEN. CEN uses APF to avoid collisions. When the remaining drone closest to the center in the cluster reaches the center, all remaining drones in the cluster stop moving.

[0118] Figure 5 The average time required to restore connectivity for different numbers of damaged drones is shown. The line with the marked points represents the average time required to achieve deterministic restoration using the distributed connectivity restoration algorithms CR-MA and CEN based on MADDPG and APF. The lower bounds of the two shaded areas represent the average time required to achieve actual restoration using CR-MA and CEN. The results show that the average time used by CR-MA is less than the average time used by CEN for both deterministic and actual restoration. Figure 6 The average total movement distance of all remaining drones under different numbers of damaged drones is shown. When CR-MA is used, the average total movement distance of all remaining drones is smaller. Especially when the number of damaged drones is small, the difference is more obvious. The above results show that CR-MA restores connectivity between disconnected clusters at a faster speed and with a shorter total movement distance.

[0119] Those skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk. Optionally, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits, and accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of software function modules. The present invention is not limited to any specific form of combination of hardware and software.

[0120] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for restoring connectivity of UAV communication network based on MADDPG and APF, characterized in that: Include: Establish a network connectivity restoration task model; Design network connectivity restoration mechanisms; Define action space, state space, and reward function; Design and train a network connectivity restoration model based on MADDPG and APF; Use the trained network connectivity restoration model to achieve network connectivity restoration.

2. The method for restoring connectivity of a UAV communication network based on MADDPG and APF according to claim 1 is characterized in that: Establishing a network connectivity restoration task model includes: The goal of connectivity recovery is to find new locations for the remaining drones in the drone cluster after removing the damaged drone, so that each cluster can be restored to connectivity as quickly as possible, while the total movement distance of the remaining drones is as short as possible; To achieve the connectivity restoration goal, it is necessary to minimize the maximum value of the movement time of all remaining drones and the total movement distance of the remaining drones.

3. The method for restoring connectivity of a drone communication network based on MADDPG and APF according to claim 1, characterized in that: The network connectivity recovery mechanism includes drone damage identification, and the specific process is as follows: First, each remaining drone processes the received message. Received u j location information; if Received u j Status information Indicates u j Normal operation; Indicates u j Damaged; otherwise Indicates u j The state of is unknown; at time t, establish The remaining drone set and the uncertain drone set; Then, each remaining UAV constructs an undirected graph G based on its position information and state information i (U i ,E i ); Finally, calculate G i (U i ,E i )’s Laplace matrix L i The eigenvalues ​​of L i The feature values ​​of are used to draw conclusions about whether there are other clusters.

4. The method for restoring connectivity of a drone communication network based on MADDPG and APF according to claim 3 is characterized in that: The network connectivity recovery mechanism includes the relocation of drones based on the leader-follower mechanism, including: (1) Restoring connectivity For each cluster, the drone closest to the cluster center is selected as the leader and the other drones are followers. Each leader moves at a maximum speed v max Flying towards the center of the cluster, each follower selects a superior drone to maintain connection with, so the entire cluster is connected; (2) Accelerate the restoration of connectivity First, from the perspective of the uncertain drones, each uncertain drone is assigned to its closest follower; if a follower has no uncertain drones to explore, but its neighbors do, then it regards all the uncertain drones that its neighbors are going to explore as its own uncertain drones to explore, and cooperates with its neighbors; Next, to find all leaders, all uncertain drones are considered as virtual leaders moving toward the center of the cluster. quilt Successful exploration means that at a certain time t, D(i,j)≤R is satisfied, where By exploring the virtual leader, it is determined whether the uncertain UAV corresponding to the virtual leader is the leader; in addition, at the exploration time t, the follower Might encounter unsure drones that work well at this time Then Can be used with All the uncertain drones in the cluster that are running well establish communication, and it is possible to know whether these uncertain drones are leaders, without having to explore their corresponding virtual leaders; (3) Reduce unnecessary movement of remaining drones Each follower closest to the uncertain UAV and its neighbors should actively explore the uncertain UAV, while other followers do not need to.

5. The method for restoring connectivity of a UAV communication network based on MADDPG and APF according to claim 4 is characterized in that: Determining whether the uncertain drone corresponding to the virtual leader is the leader includes: ① When Successful exploration at time t If it can be located at If the drone establishes communication, the drone is not sure is well run, it is the leader, Can Communicate with all followers in the cluster; ② When Successful exploration at time t , but it cannot be compared with If the drone establishes communication, the drone is not sure Instead of being a leader, it can be damaged or become a follower; In the above process, if Discover Is running well, need to The cluster and The clusters they are in are merged, the leader stops moving when it reaches the center of the cluster, and each follower stops moving when the conditions for restoring connectivity are met.

6. The method for restoring connectivity of a UAV communication network based on MADDPG and APF according to claim 1, characterized in that: Define the action space as: If the follower If the communication with the upper drone is about to be lost, or if the drone is about to collide with the surrounding drones, Use APF for path planning; otherwise, Gain speed with MADDPG.

7. The method for restoring connectivity of a drone communication network based on MADDPG and APF according to claim 1, characterized in that: Define the state space as: For local observations, The drone observed was The remaining drones and uncertain drones except the leader within the communication range; for global observation, All uncertain drones and all remaining drones were observed.

8. The method for restoring connectivity of a UAV communication network based on MADDPG and APF according to claim 1, characterized in that: The reward function is defined as the reward for completing the task of restoring connectivity, the reward for the remaining drones involved in the local observation to successfully explore an uncertain drone, and the incentive The sum of rewards for exploring uncertain drones.

9. The method for restoring connectivity of a drone communication network based on MADDPG and APF according to claim 3, characterized in that: The training process of designing a network connectivity restoration model based on MADDPG and APF is as follows: Randomly select multiple drones and put them in a damaged state; Start UAV damage identification; If the network is not connected, obtain the initial local observations and initial global observations of all remaining drones; For each damaged drone or leader that has reached the cluster center, the action is the zero vector; For each leader that has not reached the cluster center, the movement direction points to the cluster center, and the size is the maximum speed value; For each follower, if the follower is about to lose communication with the parent drone or is about to collide with the surrounding drones, use APF to obtain the speed; otherwise, use MADDPG to obtain the speed; Execute actions and obtain rewards, obtain new local and global observations; Store the last local observation, the last global observation, the velocity, the new local observation, and the new global observation in the experience replay buffer; Randomly sample and update the parameters of the policy network, value network, target policy network, and target value network from the experience replay buffer; If the remaining drones determine that connectivity is restored, the round ends.

10. A UAV communication network connectivity recovery system based on MADDPG and APF, characterized in that: include: A task model building module is used to build a network connectivity restoration task model; The recovery mechanism module is used to design the network connectivity recovery mechanism; Space definition module, used to define action space, state space and reward function; Model training module, used to design and train network connectivity restoration models based on MADDPG and APF; The connectivity restoration module is used to restore network connectivity using the trained network connectivity restoration model.

Images