Multi-unmanned aerial vehicle cooperative efficient deployment method in dynamic environment

Abstract

The invention discloses a multi-unmanned aerial vehicle cooperative efficient deployment method in a dynamic environment, and the method comprises the steps: S1, constructing a multi-target optimization model based on a genetic algorithm, and solving the multi-target optimization model, so as to plan a feasible path with the least number of relay unmanned aerial vehicle I nodes between a base station and a nest; and S2, constructing an intelligent agent control model based on a deep reinforcement learning algorithm, so that each relay unmanned aerial vehicle II can adjust a motion strategy according to the real-time position of the task unmanned aerial vehicle, the communication requirement and the environment change in a dynamic task scene to guarantee the stability of a communication link between the aircraft nest and the task unmanned aerial vehicle. According to the two-stage unmanned aerial vehicle cooperative efficient optimization deployment scheme provided by the invention, the efficiency and the stability of an air communication link are remarkably improved in a complex three-dimensional space environment.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) communication. More specifically, this invention relates to a method for the efficient collaborative deployment of multiple UAVs in dynamic environments. Background Technology

[0002] Drones are characterized by high mobility and flexible deployment. Therefore, drones can be used to carry communication payloads and serve as communication relay nodes. Through efficient drone deployment, network communication quality can be improved and network coverage expanded.

[0003] In UAV relay communication networks, current deployment strategies for relay UAVs mostly focus on the selection of a single relay node or multi-hop path planning across the entire network. Common methods include greedy algorithms and graph-based shortest path methods. These methods suffer from the following shortcomings in practical applications: First, the deployment strategies lack structure, making it difficult to quickly and efficiently establish stable links from communication base stations to mission terminals. This is especially true when mission terminals are far from communication base stations or are obstructed, resulting in poor relay link connectivity and high deployment redundancy. Second, existing methods do not uniformly consider the coupling relationship between deployment paths and multi-UAV release strategies within a single hop, leading to coarse optimization granularity and an inability to achieve fine-grained multi-scale control. Furthermore, existing methods mostly focus on the deployment of independent relay nodes, regional coverage, or topology control of static sensing networks, rarely involving deployment models with engineering feasibility and structural advantages such as "base station - optimized deployment route - multi-point release from the UAV nest."

[0004] It is evident that current technology has not yet formed a collaborative deployment system that can directly start from a base station, advance along an optimized path, and release multiple relay drones at the drone nest. In particular, it lacks a unified methodological framework that integrates communication constraints, flight energy consumption, obstacle avoidance, and multi-hop strategy optimization. That is, how to achieve the deployment of multiple relay drones from a fixed base station along an optimized path in a drone collaborative communication scenario, until they reach the drone nest, and then release multiple relay drones from the drone nest to establish the final link, thereby forming a stable and efficient communication connection structure, is the technical problem that this invention needs to solve. Summary of the Invention

[0005] One object of the present invention is to solve at least the above-mentioned problems and / or defects, and to provide at least the advantages described below.

[0006] To achieve these objectives and other advantages of the present invention, a method for efficient collaborative deployment of multiple unmanned aerial vehicles (UAVs) in a dynamic environment is provided, comprising:

[0007] S1. Construct a multi-objective optimization model based on genetic algorithm, and solve the multi-objective optimization model to plan a feasible path with the minimum number of relay UAV I nodes between the base station and the UAV nest;

[0008] S2. Construct an intelligent agent control model based on deep reinforcement learning algorithm, so that in dynamic task scenarios, each relay UAV II can adjust its motion strategy according to the real-time position, communication requirements and environmental changes of the task UAV, so as to achieve coordinated and efficient deployment of communication links between the UAV nest and the task UAV.

[0009] Preferably, in S1, the multi-objective optimization model is constructed as follows:

[0010] S10. Let the location of the base station be p0, and the location of the equipment nest be p0. M+1 The location of each relay UAV I node between the base station and the nest is p i Let i = 0, 1, ..., M, where M is the number of relay UAV I nodes, and the spatial distance between any adjacent relay UAV I nodes must satisfy the following formula:

[0011] ||p i+1 -p i ||2≤R

[0012] In the above formula, R is the maximum communication radius of the UAV;

[0013] S11. Assume that each relay UAV node I operates at a constant speed v>0 and a constant flight power P. f If a deployment mission is executed with a value greater than 0, the total flight energy consumption is represented by the following formula:

[0014]

[0015] In the above formula, E total ∝ indicates

[0016] S12. Taking the minimum path energy consumption and the minimum number of relays into the modeling considerations, the comprehensive objective function for flight energy consumption modeling is represented by the following formula:

[0017]

[0018] In the above formula, α is the relay node number penalty factor, and λ is the obstacle penalty weight. It is an indicator function, and Satisfy the following formula:

[0019]

[0020] In the above formula, Q represents the number of obstacle regions. Indicates the j-th penalty region

[0021] S13. Assume the maximum number of relays allowed by the system is M. max Therefore, the path optimization problem of relay UAV node I in the final optimization model can be comprehensively represented as:

[0022]

[0023] S14. A genetic algorithm is used as the global search method to solve the path optimization problem in S13.

[0024] Preferably, in S2, the agent control model is constructed using the Proximal Policy Optimization (PPO) algorithm, and the construction method is as follows:

[0025] S20. Let N be the reciprocal of the relay UAV II in the nest. ru The number of mission drones is N. tu Then, at time t, the states s of each relay UAV II and each mission UAV are... t It can be represented by the following formula:

[0026]

[0027] In the above formula, Let represent the three-dimensional coordinates of the i-th relay UAV II at time t. Let represent the three-dimensional coordinates of the j-th mission UAV at time t;

[0028] S21. Based on the spatial state obtained in S20, the coordinates of each relay UAV II in three-dimensional space at time t+1. It is characterized by the following formula:

[0029]

[0030] In the above formula, Δp ru Given the sequence {(0,0,v),(0,0,-v),(0,v,0),(0,-v,0),(v,0,0),(-v,0,0)}, then the action a of each relay UAV II at time t is... t It is characterized by the following formula:

[0031]

[0032] S22. Based on the coverage status of the mission UAV, the safe distance between relay UAV II and the mission UAV, and the communication distance between relay UAV II and the nest, design the following reward function r. t :

[0033] r t =R cover +R uncover +R collision,ru +R collision,ru-tu

[0034] In the above formula, R cover For mission drone coverage rewards, R uncoverPenalty for mission drone not being covered, R collision,ru For collision penalties between relay drones II, R collision,ru-tu Penalty for collisions between relay UAV II and mission UAV;

[0035] S23. Use the PPO algorithm for model training and compensate for any abnormalities in the model output results.

[0036] Preferably, in S22, the mission drone covers the reward R. cover It is characterized by the following formula:

[0037]

[0038] In the above formula, As an indicator function, when d i,j ≤R is 1, otherwise it is 0, d i,j Let β1 represent the Euclidean distance between the mission drone j and the relay drone i, and let β1 be the penalty coefficient I.

[0039] Mission drone not covered penalty R uncover It is characterized by the following formula:

[0040]

[0041] In the above formula, β2 is the penalty coefficient II;

[0042] Collision penalty is represented by the following formula:

[0043]

[0044] In the above formula, β3 and β4 are penalty coefficients III and IV, respectively, and d safe To maintain a safe distance between drones.

[0045] Preferably, in S23, the training process of the PPO algorithm consists of multiple iteration cycles, and each iteration cycle is divided into two stages: data acquisition and parameter update.

[0046] In each iteration cycle, three neural networks are set up during the network model initialization phase, including: a baseline policy network π. θold Updatable Policy Network π θ and value assessment network And π θold The parameters are kept constant throughout the entire iteration cycle, while π θ and The update phase will involve multiple optimization steps.

[0047] Preferably, during the parameter update phase, the policy network π is based on the data accumulated in the buffer.θ and value network Iterative optimization is performed according to a preset step size K;

[0048] Among them, the policy network π θ The policy update is performed using a truncation mechanism, and the objective function L is defined. CLIP as follows:

[0049]

[0050] In the formula, the clip() function updates the magnitude through the clipping coefficient ∈ constraint strategy, π θold (a t |s t ) and π θ (a t |s t ) represent the old and new strategies in state s, respectively. t Choose action a t The probability, A t It is the dominant function, and γ is the discount factor for future rewards, r t' For the immediate reward obtained at time step t', γ t′ Let be the discount factor raised to the power of time step t'.

[0051] Preferably, in S23, the abnormal cases in the model output results include:

[0052] Anomaly 1: If a negative value is found in the model's reward function output during task allocation, a relay aircraft will be added at the midpoint between the uncovered task drone location and the nearest relay aircraft II, using the following formula:

[0053] P ru_new =(P ru_closest +P tu_uncover ) / 2

[0054] In the above formula, P ru_new For the coordinates of the newly added repeater, P tu_uncover P is the coordinate of the uncovered mission drone. ru_closest The coordinates of the relay UAV II closest to the UAV on an uncovered mission.

[0055] Anomaly 1: In the relay layout output by the intelligent agent control model, if the distance between two unmanned relay drones II is detected to be less than a set threshold, it is evaluated whether there exists a location that can simultaneously cover all the mission drones originally covered by the two unmanned relay drones II, and that the original mission drone locations can be balanced from that location. If such a location exists, a new relay drone will replace the original two relay drones.

[0056] This invention offers at least the following advantages: The proposed two-stage UAV collaborative high-efficiency optimization deployment scheme significantly improves the efficiency and stability of airborne communication links in complex three-dimensional spatial environments. The first stage uses a genetic algorithm to globally optimize the path of the relay UAV, satisfying communication continuity and obstacle avoidance constraints while effectively reducing the number of relay UAVs and energy consumption, thus minimizing the overall system deployment cost. Building on this, the second stage introduces the PPO algorithm, enabling intelligent autonomous decision-making by the UAVs in dynamic mission scenarios. The PPO algorithm allows the relay UAVs to adaptively adjust their flight trajectories, ensuring continuous and stable coverage and communication support for the mission UAVs.

[0057] Other advantages, objectives and features of the present invention will become apparent in part from the following description, and in part from those skilled in the art through study and practice of the invention. Attached Figure Description

[0058] Figure 1 This is a system model diagram of the present invention;

[0059] Figure 2 This is a framework diagram of the PPO algorithm of this invention. Detailed Implementation

[0060] The present invention will now be described in further detail with reference to the accompanying drawings, so that those skilled in the art can implement it based on the description.

[0061] To achieve efficient and stable deployment of relay communication nodes in complex three-dimensional environments, this invention proposes a phased UAV collaborative optimization deployment scheme, focusing on "path planning optimization" and "intelligent control enhancement." Specifically, the first phase uses a genetic algorithm to optimize the deployment path of the relay UAVs, solving the problems of minimizing path energy consumption, minimizing the number of relays, and obstacle avoidance constraints. The second phase introduces a deep reinforcement learning algorithm, enabling the relay UAVs to adaptively adjust during mission execution, optimizing their flight decisions and ensuring mission coverage and communication stability. Specifically, this includes:

[0062] I. First Phase

[0063] like Figure 1 As shown, in the first stage, for the path planning problem of relay UAVs between base stations and UAV nests, this invention constructs a spatial multi-objective optimization model that integrates communication constraints, energy consumption constraints, obstacle avoidance constraints, and deployment costs, and uses a genetic algorithm to solve it, in order to obtain a feasible path with continuous communication, low energy consumption, and the fewest nodes. This stage mainly solves the optimization problem in the static deployment problem, ensuring that the relay link has a good initial layout and resource configuration foundation.

[0064] Specifically, to achieve efficient and stable deployment of airborne communication links, this invention focuses on the path optimization problem of relay drones between base stations and drone nests. Specifically, it considers the location of the base station... With the location of the nest between, This represents a three-dimensional Euclidean space, where M represents the number of relay drones, and several relay drone nodes are deployed. The goal is to construct a multi-hop communication link that satisfies communication constraints, obstacle avoidance limitations, and minimizes energy consumption and deployment costs. This problem is essentially a constrained path optimization problem, the core of which lies in finding a feasible, efficient, and compact spatial polyline sequence in a complex spatial environment. Let the communication link consist of a sequence of points {p0, p1, ..., p...}. M ,p M+1 As defined by}, where p0 and p M+1 These are fixed start and end nodes, with intermediate points representing the deployment locations of relay drones to be optimized. To ensure communication continuity, the system requires that the spatial distance between any adjacent nodes does not exceed the drone's maximum communication radius R > 0, i.e.:

[0065]

[0066] In terms of flight energy consumption modeling, it is assumed that all relay UAVs fly at a constant speed v>0 and a constant flight power P. f When executing a deployment mission, the total flight energy consumption of the system can be approximated as the linear sum of the distances between all adjacent nodes in the communication path, i.e.:

[0067]

[0068] Furthermore, considering that drones are a limited resource in actual deployment, their numbers are restricted and their deployment costs are high, therefore the number of relays M used should be minimized to reduce overall system overhead and deployment complexity. Based on this, this invention introduces a joint optimization objective, incorporating both "minimum path energy consumption" and "minimum number of relays" into the modeling considerations. The comprehensive objective function is expressed as:

[0069]

[0070] Where α>0 is the relay node number penalty factor, and λ>0 is the obstacle penalty weight. Assuming there are Q obstacle regions, and using the indicator function as an example, the obstacle penalty can be defined as:

[0071]

[0072] in, This represents the j-th penalty region.

[0073] To ensure task feasibility, the model also needs to introduce a limit on the maximum number of relay nodes. Let M be the maximum number of relays allowed by the system. max Then it should satisfy:

[0074] M≤M max

[0075] In summary, the final optimization model comprises three optimization objectives: minimizing path energy consumption, minimizing the number of relays, and minimizing obstacle avoidance cost, forming a mixed-integer nonlinear programming problem, as follows:

[0076]

[0077] To effectively solve the aforementioned relay UAV path optimization problem, a genetic algorithm is employed as the global search method. By simulating the selection, crossover, and mutation processes of biological evolution in nature, the genetic algorithm can search for the globally optimal or near-optimal solution in complex optimization problems. The advantage of this algorithm lies in its ability to effectively handle nonlinear, non-convex, and high-dimensional search spaces, making it suitable for solving path planning problems with multiple constraints and optimization objectives.

[0078] ①Individual coding and initialization

[0079] In genetic algorithms, each individual represents a possible solution. Since the problem involves the spatial location and number of relay drones, each individual uses a real-valued vector to encode the location sequence of the relay drones, specifically represented as:

[0080] P = [p1, p2, ..., p M ]

[0081] Where, p i Let represent the 3D position of the i-th relay UAV. The initial population is generated by randomly generating multiple individuals to ensure that the initial distribution of relay positions covers the entire search space. The initial solution can be generated through uniform randomization or local perturbation to ensure population diversity and prevent getting trapped in local optima.

[0082] ② Fitness function

[0083] The fitness function is the most crucial part of a genetic algorithm, determining the quality of each individual. In this problem, the fitness function comprehensively considers the energy consumption of the communication path, obstacle avoidance penalties, and the objective of minimizing the number of relays. Specifically, the fitness function is the negative of the objective function, i.e.:

[0084]

[0085] The first term is the total path distance, the second is the penalty for the number of relay nodes, and the third is the obstacle penalty. A lower fitness value indicates a better individual. To ensure feasibility during the search process, the fitness value of individuals that violate constraints will be increased as a penalty, thus guiding the algorithm to exclude infeasible solutions.

[0086] ③Select Operation

[0087] Selection is the core step in genetic algorithms, determining which individuals advance to the next generation. This invention employs a tournament selection method, randomly selecting multiple individuals from the current population to "compete," and then choosing the individual with the best fitness as the parent. Tournament selection ensures a high probability of selecting superior individuals while preserving appropriate diversity.

[0088] ④ Cross operations

[0089] Crossover is used to simulate gene recombination in organisms, producing new offspring by exchanging genes between two parent individuals. In this problem, crossover employs single-point crossover or interval recombination methods using real-valued vectors. After the crossover operation, the offspring will have their parent relay drone location sequences swapped based on the selected crossover point, generating new individuals. The purpose of crossover is to explore better solutions by combining the advantageous characteristics of parent individuals.

[0090] ⑤ Mutation operation

[0091] Mutation operations simulate the process of gene mutation, increasing population diversity by making minor adjustments to existing solutions. Specifically, mutation operations generate new solutions by applying Gaussian perturbations or small-scale random relocations to randomly selected relay positions within individuals. Mutation operations can prevent the population from getting trapped in local optima, ensuring global search capability.

[0092] ⑥ Elite Retention Strategy

[0093] To improve the convergence speed of the algorithm and prevent the optimal individual from being lost during iteration, this invention employs an elite retention strategy. In each generation, the individual with the best fitness directly enters the next generation without undergoing crossover and mutation operations. This ensures that the optimal solution is always retained in the population, thereby accelerating the convergence of the algorithm.

[0094] ⑦ Termination Conditions

[0095] The termination conditions of genetic algorithms are typically based on the maximum number of iterations or the stability of the fitness. In this problem, the algorithm stops when the maximum number of iterations is reached, or when the change in the optimal solution over several consecutive generations is less than a predetermined threshold. By setting appropriate termination conditions, overcomputation by the algorithm is avoided, while ensuring the quality of the solution.

[0096] II. Second Phase

[0097] Building upon the first phase, the second phase further introduces a reinforcement learning-based intelligent decision-making mechanism, employing the Proximal Policy Optimization (PPO) algorithm to achieve dynamic collaborative control of the relay UAV and the mission UAV in three-dimensional space. By constructing a state-action-reward function model, the PPO algorithm can adaptively adjust the position and behavior strategies of the relay UAV to cope with complex situations such as dynamic mission changes, coverage adjustments, and unforeseen events, thereby improving the system's mission coverage and communication stability. The two phases are functionally interconnected but strategically distinct, forming a complete aerial communication link deployment scheme integrating global path optimization and intelligent control optimization.

[0098] Specifically, building upon the initial deployment path optimization of the relay UAV in the first phase, the second phase focuses on the dynamic control and collaborative optimization of the relay UAV during mission execution. Compared to static path planning, in dynamic mission scenarios, the relay UAV needs to continuously adjust its motion strategy based on the real-time location, communication requirements, and environmental changes of the mission UAV to ensure the stability of the communication link and the overall system performance. Therefore, enabling the relay UAV to make autonomous decisions and adapt to environmental changes becomes the core challenge of this phase of research. To address this, this invention introduces a reinforcement learning method, employing the PPO algorithm to construct an intelligent agent control model, enabling the relay UAV to continuously learn and optimize its behavioral strategies through interaction with the environment. In this model, the rational design of the state space, action space, and reward function is crucial for achieving effective learning.

[0099] ①State Space

[0100] state space This includes the three-dimensional coordinates of all relay and mission UAVs in the system. Assume there are N UAVs in the system. ru A relay drone and N tu If there are 100 drones for a given mission, then the total number of drones N = N tu +N ru The state of the system at time t is represented as:

[0101]

[0102] in This represents the three-dimensional coordinates of the i-th relay UAV at time t. Let represent the three-dimensional coordinates of the j-th mission UAV at time t. The dimension of the state space is .

[0103] ② Action Space

[0104] Action space Includes seven possible discrete movement directions for each relay drone in three-dimensional space:

[0105]

[0106] Where 0 indicates that the relay drone does not move, +z and -z indicate that the relay drone moves up and down along the z-axis; +y and -y indicate that it moves forward and backward along the y-axis; and +x and -x indicate that it moves left and right along the x-axis. The coordinates for each action are updated as follows:

[0107]

[0108] Δp ru ∈{(0,0,0),(0,0,v),(0,0,-v),(0,v,0),(0,-v,0),(v,0,0),(v,0,0)}

[0109] The total dimension of the action space is Correspondingly, the system's action at time t is represented as:

[0110]

[0111] It's worth noting that if the preset repeater location exceeds the communication range of the hive, the code will pull it back to the boundary of the communication range. Specifically, it works as follows: First, it calculates the direction vector from the hive to the current repeater location. Then, it normalizes this vector and scales it according to the communication range radius to obtain the vector from the hive to the communication range boundary. Next, it adds this vector back to the hive location to calculate the repeater's new position on the communication boundary. Finally, to ensure the coordinates are integers, it uses rounding and converts the result to an integer type, thus updating the repeater's position.

[0112] ③ Reward function

[0113] The reward function is designed to take into account factors such as the coverage status of the mission drone, the safe distance between the relay drone and the mission drone, and the communication distance between the relay drone and the avionics nest, in order to achieve the following optimization objectives: maximize the number of mission drones covered by relay drones; penalize mission drones that are not covered; and avoid collisions between relay drones and mission drones.

[0114] Let the safe distance between drones be d. safe The specific reward design is as follows:

[0115] 1) Mission drone coverage reward

[0116] If the mission drone j is covered by the relay drone i, that is, the Euclidean distance d between the two is... i,j If R ≤ R, the system receives a positive reward:

[0117]

[0118] Where β1 is the penalty coefficient.

[0119] 2) Penalty for mission drone not being covered

[0120] For each mission drone, check its minimum distance to all relay drones. If mission drone j is not covered by any relay drone, i.e., all d i,j >R, the system is penalized:

[0121]

[0122] Where β2 is the penalty coefficient.

[0123] 3) Collision penalty

[0124] If the distance between the relay drone and the mission drone or the relay drone is less than the minimum safe distance d safe The system is penalized for collisions:

[0125]

[0126] Where β3 and β4 are penalty coefficients.

[0127] 4) Total Reward Function

[0128] Adding the rewards and penalties from each of the above components together, we get the total reward:

[0129] R total =R cover +R uncover +R collision,ru +R collision,ru-tu

[0130] ④ Network training

[0131] like Figure 2 As shown, the training process of the PPO algorithm consists of multiple iterative cycles, each divided into two phases: data acquisition and parameter update. The initialization phase sets up three neural networks: a baseline policy network π... θold Updatable Policy Network π θ and value assessment network It is worth noting that π θold The parameters remain constant throughout the entire period, while π θ and The update phase will involve multiple optimization steps.

[0132] During the data acquisition phase, the baseline policy network π θoldContinuously interact with the environment, and generate a state-action-reward triplet (s) for each interaction. t ,a t ,r t The data is stored in the experience buffer until the current task is completed. During the parameter update phase, the policy network π is updated based on the data accumulated in the buffer. θ and value network Iterative optimization is performed according to a preset step size K.

[0133] Updatable Policy Network π θ The parameter update uses a truncation mechanism, and its objective function is defined as:

[0134]

[0135] In the formula, the clip function updates the magnitude through the clipping coefficient ∈ constraint strategy; π θold (a t |s t ) and π θ (a t |s t ) represent the old and new strategies in state s, respectively. t Choose action a t The probability; the advantage function A t Calculated using generalized dominance estimation:

[0136]

[0137] Where γ is the discount factor for future rewards.

[0138] Value Network The optimization is achieved by minimizing the difference between the predicted value and the actual reward, and its loss function is:

[0139]

[0140] In the formula, Represents the value network in state s t The expected cumulative discounted reward.

[0141] After completing the parameter update, π θ The weights are synchronized to π. θold The simulation environment is then reset and the next training cycle begins.

[0142] ⑤ Compensation for special circumstances

[0143] Two types of anomalies in the model output and their handling strategies:

[0144] Anomaly 1: If a negative value is found in the model's reward function output during task allocation, it indicates that some task drones are not effectively covered. To address this issue, this solution takes the following measure: A relay drone is added at the midpoint between the location of the uncovered task drone and the nearest drone. This aims to expand coverage by adding relay drones, ensuring that all task drones are served. The formula is:

[0145] P ru_new =(P ru_closest +P tu_uncover ) / 2

[0146] Where P ru_new For the coordinates of the newly added repeater, P tu_uncover P represents the coordinates of the uncovered mission machine. ru_closest The coordinates are those of the relay drone closest to the uncovered mission aircraft.

[0147] An example for abnormal situation one is as follows: Suppose that in a certain task assignment, the system displays the task drone TU sample The absence of coverage by any relay drone indicates a coverage blind spot in the current relay deployment. According to the handling method for anomaly one proposed in this invention, the system locates the distance TU. sample The nearest relay drone, the coordinates of the two are P ru_closest =(20,30), TU sample = (40, 50), calculate the position of the midpoint between the two as P. ru_new = (30, 40). Therefore, the system deploys a new relay UAV at this location to achieve communication with the mission TU. sample The effective coverage expands the service range of the original relay system and ensures the integrity and reliability of the system's tasks.

[0148] Anomaly 2: If the distance between two relays in the model's output relay layout is less than a set threshold, this may indicate resource redundancy. To address this, this solution proposes the following optimization strategy: First, assess whether there exists a location that can simultaneously cover all the task machines originally covered by the two relays, and from this location, the original task machine locations can be covered in a balanced manner. If such a location exists, a new relay will replace the original two relays, thereby achieving optimal resource allocation.

[0149] The specific implementation plan for abnormal situation two is as follows:

[0150] If, during a task allocation, the system detects that the distance between two relay drones A (coordinates (10,10)) and B (coordinates (20,10)) is less than a set threshold (e.g., 20km), indicating resource redundancy, then according to the optimization strategy proposed in this invention, it assesses whether there is a new location between them that can replace their function. Finally, it selects their geometric midpoint (15,10) to deploy a new relay drone, while removing A and B. This achieves complete coverage of the original task drones and reduces one relay device, thereby improving resource utilization and distribution balance.

[0151] The above solution is merely an illustration of a preferred example and is not limited thereto. When implementing this invention, appropriate substitutions and / or modifications can be made according to the user's needs.

[0152] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. It can be applied to various fields suitable for the present invention. Other modifications can be readily made by those skilled in the art. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and examples shown and described herein.

Claims

1. A method for efficient collaborative deployment of multiple unmanned aerial vehicles (UAVs) in a dynamic environment, characterized in that, include: S1. Construct a multi-objective optimization model based on genetic algorithm, and solve the multi-objective optimization model to plan a feasible path with the minimum number of relay UAV I nodes between the base station and the UAV nest; S2. Construct an intelligent agent control model based on deep reinforcement learning algorithm, so that in dynamic task scenarios, each relay UAV II can adjust its motion strategy according to the real-time position, communication requirements and environmental changes of the task UAV, so as to achieve coordinated and efficient deployment of communication links between the UAV nest and the task UAV.

2. The method for efficient deployment of multiple UAVs in a dynamic environment as described in claim 1, characterized in that, In S1, the multi-objective optimization model is constructed as follows: S10. Let the location of the base station be p0, and the location of the equipment nest be p0. M+1 The location of each relay UAV I node between the base station and the nest is p i Let i = 0, 1, ..., M, where M is the number of relay UAV I nodes, and the spatial distance between any adjacent relay UAV I nodes must satisfy the following formula: ||p i+1 -p i ||2≤R In the above formula, R is the maximum communication radius of the UAV; S11. Assume that each relay UAV node I operates at a constant speed v>0 and a constant flight power P. f If a deployment mission is executed with a value greater than 0, the total flight energy consumption is represented by the following formula: In the above formula, E total ∝ represents the total flight energy consumption of all relay drones; S12. Taking the minimum path energy consumption and the minimum number of relays into the modeling considerations, the comprehensive objective function for flight energy consumption modeling is represented by the following formula: In the above formula, α is the relay node number penalty factor, and λ is the obstacle penalty weight. It is an indicator function, and Satisfy the following formula: In the above formula, Q represents the number of obstacle regions. This represents the j-th penalty region; S13. Assume the maximum number of relays allowed by the system is M. max Therefore, the path optimization problem of relay UAV node I in the final optimization model can be comprehensively represented as: S14. A genetic algorithm is used as the global search method to solve the path optimization problem in S13.

3. The method for efficient deployment of multiple UAVs in a dynamic environment as described in claim 1, characterized in that, In S2, the agent control model is constructed using the Proximal Policy Optimization (PPO) algorithm, and the construction method is as follows: S20. Let N be the number of relay UAVs II in the nest. ru The number of mission drones is N. tu Then, at time t, the states s of each relay UAV II and each mission UAV are... t It can be represented by the following formula: In the above formula, Let represent the three-dimensional coordinates of the i-th relay UAV II at time t. Let represent the three-dimensional coordinates of the j-th mission UAV at time t; S21. Based on the spatial state obtained in S20, the coordinates of each relay UAV II in three-dimensional space at time t+1. It is characterized by the following formula: In the above formula, Δp ru Given the sequence {(0,0,0),(0,0,v),(0,0,-v),(0,v,0),(0,-v,0),(v,0,0),(-v,0,0)}, then the action a of each relay UAV II at time t is... t It is characterized by the following formula: S22. Based on the coverage status of the mission UAV, the safe distance between relay UAV II and the mission UAV, and the communication distance between relay UAV II and the nest, design the following reward function r. t : r t =R cover +R uncover +R collision,ru +R collision,ru-tu In the above formula, R cover For mission drone coverage rewards, R uncover Penalty for mission drone not being covered, R collision,ru For collision penalties between relay drones II, R collision,ru-tu Penalty for collisions between relay UAV II and mission UAV; S23. Use the PPO algorithm for model training and compensate for any abnormalities in the model output results.

4. The method for efficient deployment of multiple UAVs in a dynamic environment as described in claim 3, characterized in that, In S22, the mission drone coverage reward R cover It is characterized by the following formula: In the above formula, As an indicator function, when d i,j ≤R is 1, otherwise it is 0, d i,j Let β1 represent the Euclidean distance between the mission drone j and the relay drone i, and let β1 be the penalty coefficient I. Mission drone not covered penalty R uncover It is characterized by the following formula: In the above formula, β2 is the penalty coefficient II; Collision penalty is represented by the following formula: In the above formula, β3 and β4 are penalty coefficients III and IV, respectively, and d safe To maintain a safe distance between drones.

5. The method for efficient deployment of multiple UAVs in a dynamic environment as described in claim 3, characterized in that, In S23, the training process of the PPO algorithm consists of multiple iteration cycles, and each iteration cycle is divided into two stages: data acquisition and parameter update. In each iteration cycle, three neural networks are set up during the network model initialization phase, including: a baseline policy network π. θold Updatable Policy Network π θ and value assessment network And π θold The parameters are kept constant throughout the entire iteration cycle, while π θ and The update phase will involve multiple optimization steps.

6. The method for efficient deployment of multiple UAVs in a dynamic environment as described in claim 5, characterized in that, During the parameter update phase, the policy network π is based on the data accumulated in the buffer. θ and value network Iterative optimization is performed according to a preset step size K; Among them, the policy network π θ The policy update is performed using a truncation mechanism, and the objective function L is defined. CLIP as follows: In the formula, the clip() function updates the magnitude through the clipping coefficient ∈ constraint strategy, π θold (a t |s t ) and π θ (a t |s t ) represent the old and new strategies in state s, respectively. t Choose action a t The probability, A t It is the dominant function, and γ is the discount factor for future rewards, r t' For the immediate reward obtained at time step t', γ t′ Let be the discount factor raised to the power of time step t'.

7. The method for efficient deployment of multiple UAVs in a dynamic environment as described in claim 1, characterized in that, In S23, the abnormal cases in the model output include: Anomaly 1: If a negative value is found in the model's reward function output during task allocation, a relay aircraft will be added at the midpoint between the uncovered task drone location and the nearest relay aircraft II, using the following formula: P ru_new =(P ru_closest +P tu_uncover ) / 2 In the above formula, P ru_new For the coordinates of the newly added repeater, P tu_uncover P is the coordinate of the uncovered mission drone. ru_closest The coordinates of the relay UAV II closest to the UAV on an uncovered mission. Anomaly 1: In the relay layout output by the intelligent agent control model, if the distance between two unmanned relay aircraft II is less than a set threshold, it is evaluated whether there is a location that can simultaneously cover all the mission drones originally covered by the two unmanned relay aircraft II, and that can balance the original mission drone locations from that location. If such a location exists, a new relay aircraft is used to replace the original two relay aircraft.

Images