Multi-UAV (unmanned aerial vehicle) trajectory planning and data acquisition method based on multi-traveler problem

Abstract

The invention discloses a multi-UAV trajectory planning and data acquisition method based on a multi-traveler problem, and belongs to the technical field of UAV collaborative trajectory planning. The method comprises the steps that a plurality of sensors and cluster head nodes are arranged in a scene, the sensors transmit data to the cluster head nodes for storage, and the unmanned aerial vehicle obtains the cluster head node data; establishing a problem model; converting the problem model into a multi-traveler problem; performing global planning on the multi-unmanned aerial vehicle trajectory; utilizing multi-agent reinforcement learning to plan a multi-unmanned aerial vehicle local track and acquiring data; and making an unmanned aerial vehicle flight strategy to adjust the flight angle, trajectory and calculation method of the unmanned aerial vehicle in real time, and completing multi-unmanned aerial vehicle trajectory planning, data transmission and calculation tasks. According to the method, the data is acquired through the multi-unmanned aerial vehicle assisted large-scale sensor system, and rough global planning is performed on the multi-unmanned aerial vehicle trajectory by the improved genetic algorithm, so that the flight time of the unmanned aerial vehicle is shortened and the cost of the unmanned aerial vehicle is reduced while the data is acquired as much as possible, and the data acquisition rate is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle collaborative trajectory planning, and in particular relates to a multi-unmanned aerial vehicle trajectory planning and data acquisition method based on a multi-traveling salesman problem. Background Art

[0002] As the "sensory system" of the Internet of Things, sensor networks have become an important digital bridge connecting objects and people with their ubiquitous and flexible topology. Billions of sensor devices will be distributed in large-scale sensor scenarios in the future, making it possible for the Internet of Things technology to extend from industrial automation to smart transportation, smart homes to smart cities.

[0003] In response to the above needs, in order to more effectively collect data from these devices and rely on the multi-dimensional intelligent collaborative services provided by 6G networks, the emergence of high-precision drones equipped with infrared radars, high-precision sensors and high-definition cameras has provided researchers with new solutions. In addition, replacing a single drone system with a multi-drone system can increase spatial coverage, data collection and reduce resource overhead at a lower overall cost. Some people believe that the use of multiple drones in collaboration will further enhance the coverage of perception and communication in providing integrated interawareness services. Others believe that game theory can be used to model and balance the execution time and energy cost of drone-based mobile edge computing systems.

[0004] However, most of the current technologies focus on data perception, resource allocation or computational offloading, and drone trajectory planning also plays a vital role in the performance of data collection. Some people have solved the problem of multi-task vehicle group perception by designing a deep reinforcement learning-based method that can maximize energy efficiency when evaluating perception tasks in dynamic environments. The current environmental modeling problem is becoming more and more complex with the development of the physical world, and a single agent is unlikely to be the preferred solution for performing tasks.

[0005] To this end, multi-agent reinforcement learning came into being. The useful information path planning method of multi-agent reinforcement learning is used to maximize the data harvested without the need for communication between agents.

[0006] Therefore, solving the problem of minimizing the total working time of drones and balancing the service time of each drone is the technical problem that the present invention aims to solve. Summary of the invention

[0007] The purpose of the present invention is to provide a multi-UAV trajectory planning and data acquisition method based on the multi-traveling salesman problem to solve the problems raised in the above background technology.

[0008] The object of the present invention is achieved by: a multi-UAV trajectory planning and data acquisition method based on a multi-traveling salesman problem, characterized in that: the method comprises the following steps:

[0009] Step S1: multiple sensors and cluster head nodes are arranged in the scene, the sensors transmit data to the cluster head node for storage, and the drone obtains the cluster head node data;

[0010] Step S2: Establish a problem model, taking into account the clustering of sensors and the location of cluster head nodes, the set of clusters served by each drone, and the order of visiting these location nodes;

[0011] Step S3: transform the problem model into a multiple traveling salesman problem;

[0012] Step S4: using an improved genetic algorithm to perform global planning of multiple UAV trajectories;

[0013] Step S5: Using multi-agent reinforcement learning to plan the local trajectories and data acquisition of multiple UAVs;

[0014] Step S6: Use the COMA algorithm to formulate the UAV flight strategy to adjust the UAV's flight angle, trajectory and calculation method in real time to complete the multi-UAV trajectory planning, data transmission and calculation tasks.

[0015] Preferably, in step S1, multiple sensors and cluster head nodes are arranged in the scene, the sensors transmit data to the cluster head node for storage, and the drone obtains the cluster head node data, specifically:

[0016] Se sensors are randomly distributed in the scene. Each sensor collects information about the ground environment around it. The types of data collected by the sensor include humidity, temperature, noise, smoke concentration, and video data;

[0017] K cluster head nodes are deployed in the scene. K cluster head nodes pre-collect the sensors se within their collection range and wait for the drones to perform data acquisition tasks, satisfying K>>U, where U is the total number of drones;

[0018] Assume that the transmission power of the cluster head node is P k , the transmission power of other nodes in the cluster is P se , P k >>P se ;

[0019] All sensor nodes are divided into K clusters. Each sensor must be bound to a cluster head node k. The received signal-to-noise ratio of cluster head node k is expressed as:

[0020]

[0021] Among them, d k,se is the distance between cluster head node k and sensor se; α is the path loss exponent, σ 2 is the noise power.

[0022] Preferably, the problem model is established in step S2, specifically:

[0023] Step S2-1: Establishing a kinematic model of the UAV to constrain the speed, angle and distance of the data acquired by the UAV during flight;

[0024] Step S2-2: Establish a transmission channel model and analyze the average path loss of the transmission channel between the UAV and the cluster head node;

[0025] Step S2-3: Establish a UAV data acquisition model to analyze the data acquisition rate, data acquisition time, and data acquisition energy consumption of the UAV during flight;

[0026] Step S2-4: Establishing a UAV calculation model, analyzing the calculation mode of the UAV after acquiring data, and analyzing the time composition of the UAV in a data acquisition task cycle;

[0027] Step S2-5: Establish a model for the problem of the amount of data that the drone needs to obtain from the cluster head node, optimize the clustering of the drone service, the total amount of data that the drone needs to obtain from each cluster during one mission cycle, the location of the cluster head node served by the drone, the order in which the drone visits these locations, the discretized location of the flight trajectory when the drone obtains data, and the communication calculation strategy adopted by the drone.

[0028] Preferably, in step S2-4, a drone computing model is established to analyze the computing mode of the drone after acquiring the data, specifically:

[0029] Assume that the server has a total computing resource C total , let C u represents the computing resources allocated to UAV u, then the UAV computing model is:

[0030]

[0031] in, Indicates the data acquisition time, Indicates the acquisition rate of the drone; when the drone completes a round of data acquisition tasks, the data acquisition tasks are placed in the local queue for calculation;

[0032] Each drone is equipped with a CPU as a computing device. Due to its limited capabilities, it needs to transmit data to nearby drones through D2D for collaborative computing to achieve better performance;

[0033] Establish a D2D communication model. The specific operation is as follows:

[0034] Assume that the communication radius of the UAV is r. Only when the distance between two UAVs is within r can they communicate. Use N u ={n|n∈U, d n,u <r} to represent the set of potential communicable UAVs of UAV u. The transmission rate between UAV u and UAV n is:

[0035]

[0036] Among them, is the channel bandwidth allocated by the UAV for D2D communication, is the transmission power from UAV u to UAV n, is the channel gain between UAV u and UAV n, and d n,u represents the distance between UAV u and UAV n;

[0037] The transmission time is:

[0038]

[0039] Among them, η u represents the proportion of the computing tasks of D2D communication, and η u ∈[0,1]; The energy consumption of data transmission when choosing the D2D communication mode is:

[0040]

[0041] The computing time is:

[0042]

[0043] Among them, represents the local CPU frequency of UAV n; Then the total time of D2D communication computing transmission is

[0044] The energy consumption is:

[0045]

[0046] Among them, δ n (f n ) 2 is the energy consumption per unit CPU cycle of UAV n; Then the total energy consumption of D2D communication computing transmission is

[0047] Therefore, the weighted sum of the time and energy consumption of UAV u choosing the D2D communication mode is:

[0048]

[0049] Among them, θ t and θ e Respectively represent the time weighting coefficient and energy consumption weighting coefficient, satisfying θ t +θ e =1;

[0050] Assuming that drone u chooses local computing, the time is:

[0051]

[0052] in, represents the local CPU frequency of drone u, (1-η u ) is the proportion of local computing data acquisition tasks;

[0053] Energy consumption is:

[0054]

[0055] Among them, δ u (f u ) 2 is the energy consumption per unit CPU cycle of drone u; the weighted sum of time and energy consumption is:

[0056]

[0057] Preferably, in step S2-5, a data volume problem model for the drone to obtain the cluster head node is established, specifically:

[0058]

[0059] in, is the transmission time, is the calculation time, T u It is the time it takes for the drone to complete a round of data acquisition tasks. The time it takes for the drone to acquire data during flight, the time it takes for the drone to transmit data between drones after acquiring the data, and the time it takes for the drone to perform local calculations and D2D communication calculations. The time it takes for the drone to complete a round of data acquisition tasks does not exceed the flight time it takes for the drone to traverse the cluster head nodes.

[0060]

[0061] in, The data acquisition time between UAV u and UAV n; is the data transmission time between UAV u and UAV n; The data calculation time between UAV u and UAV n when the D2D communication mode is selected; Calculate time locally for the drone; The flight time of the drone.

[0062] Preferably, in step S3, the problem model is transformed into a multiple traveling salesman problem, specifically:

[0063] From the expression T(u), we know that we need to optimize the clustering of sensors, the location of cluster head nodes, the set of service clusters for each drone, and the order of visiting these location nodes to minimize the total working time of the drone and balance the service time of each drone. The optimization problem is expressed as follows:

[0064]

[0065] Among them, M u The set of clusters serving the u-th drone, Indicates the total amount of data that the drone needs to obtain from each cluster during one mission cycle; The location of the cluster head node serving the drone, is the set of discretized positions of the flight trajectory when the drone acquires data; u is drone u, and n is drone n; Indicates that the drone chooses local computing. Indicates that D2D transmission is used to perform calculations on drones. Each drone can only choose one calculation strategy, that is, γ k is the received signal-to-noise ratio of cluster head node k; is the threshold of the receiving signal-to-noise ratio; P k is the transmission power of the cluster head node; ave (dB) is the average path loss; Energy consumption for drone flight; Communication bandwidth for UAVs to obtain cluster head node data; The channel bandwidth allocated to the drone for D2D communication; d min The minimum safe distance between a drone and an obstacle and between two drones; is the center coordinate of the ob-th obstacle, P th is the threshold value; is the starting position of UAV u; B u is the bandwidth resource allocated to UAV u; B max is the maximum value of bandwidth resources;

[0066] Constraint C1 means that the received signal-to-noise ratio of the cluster head node must be greater than the threshold, and constraint C2 means that the received power of the drone during flight must be greater than the receiving threshold; constraint C3 means that the elements within the cluster do not intersect, and C4 means that the cluster sets served by each drone do not intersect either; constraint C5 means that all drones must return to the starting position for charging after acquiring data; constraint C6 means that the time it takes for a drone to complete a round of data acquisition tasks does not exceed the flight time of the drone traversing the cluster head nodes; constraint C7 means that after a cycle, the difference between the longest working time and the shortest working time of the drone must be less than the threshold ε; constraint C8 means that each drone can only choose one computing strategy, and constraint C9 means that the energy consumption of the drone to complete a round of data acquisition tasks must be less than the drone's own power; constraint C10 is a bandwidth constraint for drones used for data acquisition and D2D communication; constraint C11 is to ensure the feasibility of drone flight, and constraint C12 is to ensure the safety of drone flight.

[0067] Preferably, in step S4, an improved genetic algorithm is used to perform global planning on the trajectories of multiple UAVs, specifically:

[0068] According to the number of drones, the MTSP problem is decomposed into independent TSP sub-problems, and the independent TSP sub-problems are solved by improving the genetic algorithm through 2-opt optimization. Specifically:

[0069] Step S4-1: Initialize the population;

[0070] The population is binary-encoded, that is, the genotype of each individual is a binary-encoded symbol string;

[0071] Step S4-2: Introduce the 2-opt optimization operator to optimize the inferior individuals separately before executing each genetic operation;

[0072] In order to maintain the diversity of the population, new individuals will be randomly generated by the remaining Doping operators and directly added to the next generation population.

[0073] Step S4-3: Calculate the fitness function;

[0074] The inverse of the obtained drone task completion time function is used as a function to evaluate the individual fitness. The shorter the time, the higher the fitness, and the longer the time, the lower the fitness.

[0075] Assume that the total number of individuals in the population is N, and the fitness of drone u is Among them, T(u) represents the time it takes for a drone to complete a round of data acquisition tasks; the probability of drone u being selected is

[0076] Step S4-4: Determine whether the termination condition is met, if so, execute step S5, if not, execute step S4-5;

[0077] Step S4-5: For the population that does not meet the termination condition, perform selection operation, crossover operation and mutation operation, and return to step S4-2 after the operation until the termination condition is met.

[0078] Preferably, in step S5, multi-agent reinforcement learning is used to plan the local trajectories and data acquisition of multiple drones, specifically:

[0079] The multi-UAV sensing and computation problem is modeled as a partially observable Markov decision process, defined as a tuple in, is the system state space, is the action space, is the state transfer function, is the system observation space, is a state-action pair, r is a reward function, γ is a discount factor, Ω is an observation function,

[0080] Each agent obtains a Observed According to their strategy u (a t |j t )Select an action As a result of the action, the agent will receive a reward and the next observation from the environment; the observation function Each agent has an action observation history Then their strategy is All agents share the same reward function The discount factor for future returns is γ∈[0,1), then the discounted feedback benefit is The joint strategy of the agents produces a value function, which is the expectation of discounted feedback, expressed as And the action value function The advantage function is F π (s t ,a t )=Q π (s t ,a t )-V π (s t );

[0081] The state space, observation space, action space and reward function are specifically defined as follows:

[0082] State Space S k is a collection of time-varying task information, and the state of each cluster head node is represented by a tuple express, represents the location coordinates of the cluster head node, C u represents the CPU cycles required to complete the data acquisition task calculation, D u It indicates the total amount of data that the UAV needs to obtain from each cluster during one mission cycle; S n It is used as a D2D communication drone; S u is the state of the drone performing the data acquisition task;

[0083] Observation space: The observation space of the agent is a subset of the state space. is the set K′ (K′≤K) of the most recent mission states within the observation range of the UAV; The set U′ (U′≤U) representing the most recent UAV state;

[0084] Action space: Based on the strategy and the observed system state, each UAV performs a joint action including data acquisition, transmission and calculation in each time slot; action space d u,k is the flight distance of the drone, and ω u are the pitch angle and yaw angle of the drone, η u and N u are respectively the task ratio and selection sequence number of the UAV for D2D communication, η u and N u This constitutes the computational strategy of UAV u;

[0085] Reward function: For each state, the agent chooses their action and receives a reward from the environment, converting the state-action pair As input function; the reward of the system

[0086] in, is the gain obtained by UAV u in acquiring data at time slot t, represents the total cost of time and energy consumption of UAV u, for punishment;

[0087]

[0088] Among them, χ u,k ∈{0,1} indicates whether the data of cluster head node k is first obtained by drone u; if so, then χ u,k =1, otherwise χ u,k =0; g(D u ,Cu ) is the amount of data D obtained by UAV u u Instead of calculating the energy C u Related gain function; penalty factors are set for the two situations where the drone flies out of the target area and does not obtain enough data It is expressed as:

[0089]

[0090] in, Indicates that the drone u flies out of the target area, otherwise τ fly is the penalty value for the drone flying out of the area; Indicates the amount of data not acquired in the time slot, τ ina The penalty factor indicating the amount of data not obtained;

[0091] It is expressed as the sum of time and energy consumption, namely:

[0092]

[0093] in, and Indicates the weight of prioritizing time or energy consumption, It represents the gain obtained by UAV u when acquiring data at time t.

[0094] Preferably, in step S6, the COMA algorithm is used to formulate the UAV flight strategy to adjust the flight angle, trajectory and calculation method of the UAV in real time, and complete the multi-UAV trajectory planning, data transmission and calculation tasks, specifically:

[0095] A centralized critic is used to calculate an agent-specific advantage function to solve the confidence allocation problem in multi-agent collaborative systems. The Q(s t ,a t ) is estimated in the concentrated state s t The conditional action a t Q value;

[0096] For each agent u, an advantage function can be calculated to convert the current action Q-value and marginalization , while keeping the actions of other agents a -u Fixed, expressed as follows:

[0097]

[0098] Among them, -u represents the agents other than agent u, Fu (s t ,a t ) Compute a separate baseline for each agent that uses a centralized critic to reason about counterfactuals, where only the action of u changes, learning directly from the agent’s experience rather than relying on additional simulations, designed default actions, or reward models;

[0099] To optimize π u (·|j u ), use the policy gradient theorem, and minimize:

[0100]

[0101] Compared with the prior art, the present invention has the following improvements and advantages:

[0102] 1. Acquire data through multi-UAV-assisted large-scale sensor systems, comprehensively consider the clustering of sensors, the location of cluster head nodes, the set of clusters served by each UAV, and the order of visiting these location nodes, minimize the total working time of the UAV, and balance the service time of each UAV by balancing the amount of data in each cluster.

[0103] 2. By improving the genetic algorithm, a rough global planning of the trajectories of multiple UAVs is performed to reduce the flight time and cost of the UAVs while obtaining as much data as possible; the local trajectories between adjacent data acquisition positions in the coarse-grained global trajectory of each UAV are fine-tuned to improve the data acquisition rate.

[0104] 3. Use the advantage function based on differential rewards to solve the credit allocation problem of the intelligent agent, complete data transmission and calculation tasks while planning the trajectories of multiple drones, and effectively alleviate the load pressure of a single drone. BRIEF DESCRIPTION OF THE DRAWINGS

[0105] Figure 1 The figure is an overall flow chart of the method of the present invention.

[0106] Figure 2 Schematic diagram of the service scenario deployed in the present invention.

[0107] Figure 3 Algorithm flow chart of the improved genetic algorithm combined with the counterfactual multi-agent policy gradient algorithm.

[0108] Figure 4 This is a comparison chart of the data volume in each cluster between the traditional k-means clustering algorithm and the balanced data volume clustering algorithm.

[0109] Figure 5 This is a comparison chart of the overall running time of the genetic algorithm, the COMA algorithm and the algorithm proposed in the present application.

[0110] Figure 6 This is a comparison chart of data acquisition rates among the genetic algorithm, COMA algorithm and the algorithm proposed in the present application.

[0111] Figure 7 The following is a comparison chart of the performance of different computing strategies. DETAILED DESCRIPTION

[0112] The present invention is further summarized below with reference to the accompanying drawings.

[0113] like Figure 1 , 2 As shown, a multi-UAV trajectory planning and data acquisition method based on a multi-traveling salesman problem includes the following steps:

[0114] Step S1: Multiple sensors and cluster head nodes are arranged in the scene. The sensors transmit data to the cluster head node for storage, and the drone obtains the cluster head node data. Specifically:

[0115] Se sensors are randomly distributed in the scene. Each sensor collects information about the ground environment around it. The types of data collected by the sensor include humidity, temperature, noise, smoke concentration, and video data;

[0116] Assuming that the drone and the sensor device communicate point-to-point, due to the randomness of the sensor device location, this will not only cause additional power consumption when the drone acquires data, but also cannot guarantee that all data in the environment are collected; considering that the sensor device itself has low power and no ability to store data, K cluster head nodes are deployed in the scene. The K cluster head nodes pre-collect the sensors se within their collection range and wait for the drone to perform the data acquisition task, satisfying K>>U, where U is the total number of drones; since the transmission power of each cluster head node is different, each cluster head node has a certain acquisition range, and the drone can only acquire data within the acquisition range of the cluster head node;

[0117] Assume that the transmission power of the cluster head node is P k , the transmission power of other nodes in the cluster is P se , P k >>P se ;

[0118] All sensor nodes are divided into K clusters. Each sensor must be bound to a cluster head node k. The received signal-to-noise ratio of cluster head node k is expressed as:

[0119]

[0120] Among them, d k,seis the distance between cluster head node k and sensor se; α is the path loss exponent, σ 2 is the noise power; for cluster head node k, its received signal-to-noise ratio needs to meet the threshold Only then can the data transmitted by other sensors be accurately received.

[0121] Step S2: Establish a problem model, taking into account the clustering of sensors and the location of cluster head nodes, the set of clusters served by each drone, and the order of visiting these location nodes;

[0122] Step S2-1: Establishing a kinematic model of the UAV to constrain the speed, angle and distance of the data acquired by the UAV during flight;

[0123] Assume drone It has the ability to obtain data and calculate task requirements. The calculation data acquisition task is expressed as <D u ,C u ,T u >, D u is the total amount of data that the drone needs to obtain in one mission cycle, C u is the computing resource for obtaining data, T u It is the time it takes for the drone to complete a round of data acquisition tasks;

[0124] The kinematic model of the drone flying in an environment without considering wind and air resistance is as follows:

[0125]

[0126] Among them, (x u (t),y u (t),z u (t)) is the position coordinate of the UAV u at time t, v u (t),ω u (t) They are the speed, yaw angle and pitch angle of UAV u at time t; are the horizontal and vertical components of the acceleration of the UAV u at time t, is the derivative of x; the speed, pitch angle and yaw angle of the UAV u at time t should also satisfy the following constraints:

[0127]

[0128] where v min , v max are the minimum and maximum speeds of the drone;

[0129] The UAV searches for the optimal trajectory to reach the cluster head node in a three-dimensional scene, where there are multiple optional cluster head nodes. The coordinates of the optional cluster head node are (x k ,y k ,z k ), the distance from UAV u to cluster head node k is:

[0130]

[0131] Due to the complexity of the environment, drones do not fly at the same level. In the process of drone flight trajectory planning, drones not only need to avoid environmental obstacles, but also need to avoid collisions between drones. Obstacles in the environment can be represented by ∈={∈1,∈2,…,∈ ob ,…,∈ Ob} represents that the center coordinates of the obth obstacle are is its height;

[0132] In order to achieve the feasibility and safety of drone flights, the following conditions need to be met:

[0133]

[0134] Among them, d min is the minimum safe distance between the drone and obstacles and between two drones; when the distance d between the drone and the cluster head node is u,k ≤d th When d th is the target distance threshold.

[0135] Step S2-2: Establish a transmission channel model and analyze the average path loss of the transmission channel between the UAV and the cluster head node;

[0136] The communication between the drone and the cluster head node reuses the 5G communication frequency band, and co-frequency interference may occur between the links; there are LoS links and non-line of sight (NLoS) links between the drone and the cluster head node. The path loss of the LoS and NLoS links is:

[0137]

[0138] Among them, d u,k is the distance between the u-th UAV and the k-th cluster head node, η LoS and η NLoS are the average extra path losses for LoS links and NLoS links respectively, and f c is the carrier frequency, c is the speed of light;

[0139] The connection probability of the line-of-sight link between the UAV and the cluster head node is expressed as:

[0140]

[0141] Among them, both parameters a and b are related to the environment, h u is the flight altitude of the drone; similarly, the connection probability of the NLoS link is Γ NLoS =1-Γ LoS ; Considering only the average path loss, it is obtained by the following formula:

[0142]

[0143] in,

[0144] Step S2-3: Establish a UAV data acquisition model to analyze the data acquisition rate, data acquisition time, and data acquisition energy consumption of the UAV during flight;

[0145] When acquiring data, the drone first needs to fly to the target location in a certain direction and distance, which generates flight time. and energy consumption

[0146] The flight time is:

[0147] The corresponding energy consumption during flight is expressed as:

[0148] in, is the flight power;

[0149] According to step S2-2, the received power of UAV u is P k -ζ ave (dB), in order to ensure that the drone can receive data normally, the receiving power at the receiving end needs to be greater than the threshold P th , the channel gain from cluster head node k to drone u is expressed as:

[0150]

[0151] Among them, d u,k represents the distance from cluster head node k to UAV u;

[0152] The achievable data acquisition rate can be obtained from the received power:

[0153]

[0154] in, is the communication bandwidth used by UAV u to obtain cluster head node data, is the cumulative interference of other cluster head nodes to the communication link of UAV u, pu is the transmitting power of the UAV, expressed as:

[0155]

[0156] Among them, I k′,u It is an indicator function of whether other cluster head nodes interfere with the link between UAV u and cluster head node k:

[0157]

[0158] g k′,u is the channel gain from the interference cluster head node k′ to the UAV u, which can be expressed as:

[0159]

[0160] Among them, d u,k′ It represents the distance from the interference cluster head node k′ to the UAV u at time t;

[0161] Since a large amount of sensor data needs to be acquired in the research scenario, data must also be acquired during the flight. The data acquisition time It can be expressed as:

[0162]

[0163] in, Indicates the acquisition rate of the drone; β u,k ∈{0,1} is a binary variable, indicating whether the cluster head node k is within the acquisition range of the drone u. If so, β u,k =1; otherwise, β u,k =0; the unit energy consumption cost of data acquisition is It means that the energy consumption of data acquisition is:

[0164]

[0165] Step S2-4: Establishing a UAV calculation model, analyzing the calculation mode of the UAV after acquiring data, and analyzing the time composition of the UAV in a data acquisition task cycle;

[0166] Assume that the server has a total computing resource C total , let C u represents the computing resources allocated to UAV u, then the UAV computing model is:

[0167]

[0168] in, Indicates the data acquisition time, Indicates the acquisition rate of the UAV; after the UAV completes a round of data acquisition tasks, the data acquisition tasks are placed in the local queue for calculation;

[0169] Each UAV is equipped with a CPU as a computing device. Due to its limited capabilities, it needs to be transmitted to nearby UAVs through D2D for collaborative computing to obtain better performance;

[0170] Establish a D2D communication model. The specific operation is as follows:

[0171] Assume that the communication radius of the UAV is r. Only UAVs within a distance of r can communicate with each other. Use N u ={n|n∈U,d n,u <r} to represent the set of potential communicable UAVs of UAV u. The transmission rate between UAV u and UAV n is:

[0172]

[0173] Among them, is the channel bandwidth allocated by the UAV for D2D communication, is the transmission power from UAV u to UAV n, is the channel gain between UAV u and UAV n, and d n,u represents the distance between UAV u and UAV n;

[0174] The transmission time is:

[0175]

[0176] Among them, η u represents the proportion of the computing tasks of D2D communication, and η u ∈[0,1]; The energy consumption of data transmission when choosing the D2D communication method is:

[0177]

[0178] The calculation time is:

[0179]

[0180] Among them, represents the local CPU frequency of UAV n; Then the total time of D2D communication computing transmission is

[0181] The energy consumption is:

[0182]

[0183] Among them, δ n (f n) 2 is the energy consumption per unit CPU cycle of drone n; the total transmission energy consumption calculated by D2D communication is

[0184] Therefore, the weighted sum of time and energy consumption when drone u chooses D2D communication mode is:

[0185]

[0186] Among them, θ t and θ e Respectively represent the time weighting coefficient and energy consumption weighting coefficient, satisfying θ t +θ e =1;

[0187] Assuming that drone u chooses local computing, the time is:

[0188]

[0189] in, represents the local CPU frequency of drone u, (1-η u ) is the proportion of local computing data acquisition tasks;

[0190] Energy consumption is:

[0191]

[0192] Among them, δ u (f u ) 2 is the energy consumption per unit CPU cycle of drone u; the weighted sum of time and energy consumption is:

[0193]

[0194] Step S2-5: Establish a problem model for the amount of data that the drone acquires from the cluster head node, optimize the clustering of the drone service, the total amount of data that the drone needs to acquire from each cluster within one mission cycle, the location of the cluster head node served by the drone, the order in which the drone visits these locations, the discretized location of the flight trajectory when the drone acquires data, and the communication calculation strategy adopted by the drone, specifically:

[0195]

[0196] in, is the transmission time, is the calculation time, T uIt is the time it takes for the drone to complete a round of data acquisition tasks. The time it takes for the drone to acquire data during flight, the time it takes for the drone to transmit data between drones after acquiring the data, and the time it takes for the drone to perform local calculations and D2D communication calculations. The time it takes for the drone to complete a round of data acquisition tasks does not exceed the flight time it takes for the drone to traverse the cluster head nodes.

[0197]

[0198] in, The data acquisition time between UAV u and UAV n; is the data transmission time between UAV u and UAV n; The data calculation time between UAV u and UAV n when the D2D communication mode is selected; Calculate time locally for the drone; The flight time of the drone.

[0199] In step S3, the problem model is transformed into a multi-traveling salesman problem, specifically:

[0200] From the expression T(u), we know that we need to optimize the clustering of sensors, the location of cluster head nodes, the set of service clusters for each drone, and the order of visiting these location nodes to minimize the total working time of the drone and balance the service time of each drone. The optimization problem is expressed as follows:

[0201]

[0202]

[0203] Among them, M u The set of clusters serving the u-th drone, Indicates the total amount of data that the drone needs to obtain from each cluster during one mission cycle; The location of the cluster head node serving the drone, is the set of discretized positions of the flight trajectory when the drone acquires data; u is drone u, and n is drone n; Indicates that the drone chooses local computing. Indicates that D2D transmission is used to perform calculations on drones. Each drone can only choose one calculation strategy, that is, γ k is the received signal-to-noise ratio of cluster head node k; is the threshold of the receiving signal-to-noise ratio; P k is the transmission power of the cluster head node; ave (dB) is the average path loss; Energy consumption for drone flight; Communication bandwidth for UAVs to obtain cluster head node data; The channel bandwidth allocated to the drone for D2D communication; d min The minimum safe distance between a drone and an obstacle and between two drones; is the center coordinate of the ob-th obstacle, P th is the threshold value; is the starting position of UAV u; B u is the bandwidth resource allocated to UAV u; B max is the maximum value of bandwidth resources;

[0204] Constraint C1 means that the received signal-to-noise ratio of the cluster head node must be greater than the threshold, and constraint C2 means that the received power of the drone during flight must be greater than the receiving threshold; constraint C3 means that the elements within the cluster do not intersect, and C4 means that the cluster sets served by each drone do not intersect either; constraint C5 means that all drones must return to the starting position for charging after acquiring data; constraint C6 means that the time it takes for a drone to complete a round of data acquisition tasks does not exceed the flight time of the drone traversing the cluster head nodes; constraint C7 means that after a cycle, the difference between the longest working time and the shortest working time of the drone must be less than the threshold ε; constraint C8 means that each drone can only choose one computing strategy, and constraint C9 means that the energy consumption of the drone to complete a round of data acquisition tasks must be less than the drone's own power; constraint C10 is a bandwidth constraint for drones used for data acquisition and D2D communication; constraint C11 is to ensure the feasibility of drone flight, and constraint C12 is to ensure the safety of drone flight.

[0205] In step S4, the improved genetic algorithm is used to perform global planning of multiple UAV trajectories, specifically:

[0206] According to the number of drones, the MTSP problem is decomposed into independent TSP sub-problems, and the independent TSP sub-problems are solved by improving the genetic algorithm through 2-opt optimization. Specifically:

[0207] like Figure 3 As shown, step S4-1: initializing the population;

[0208] The population is binary-encoded, that is, the genotype of each individual is a binary-encoded symbol string;

[0209] Step S4-2: Introduce the 2-opt optimization operator to optimize the inferior individuals separately before executing each genetic operation;

[0210] In order to maintain the diversity of the population, new individuals will be randomly generated by the remaining Doping operators and directly added to the next generation population.

[0211] Step S4-3: Calculate the fitness function;

[0212] The inverse of the obtained drone task completion time function is used as a function to evaluate the individual fitness. The shorter the time, the higher the fitness, and the longer the time, the lower the fitness.

[0213] Assume that the total number of individuals in the population is N, and the fitness of drone u is Among them, T(u) represents the time it takes for a drone to complete a round of data acquisition tasks; the probability of drone u being selected is

[0214] Step S4-4: Determine whether the termination condition is met, if so, execute step S5, if not, execute step S4-5;

[0215] Step S4-5: For the population that does not meet the termination condition, perform selection operation, crossover operation and mutation operation, and return to step S4-2 after the operation until the termination condition is met;

[0216] Selection operation: The roulette method and the Doping operator are combined in a ratio of 9:1, which can avoid premature population growth while ensuring the dominant characteristics of individuals within the population; in roulette, the value of the fitness function is positively correlated with the probability of an individual being selected.

[0217] When performing the crossover operation, in order to avoid the computational burden brought by conflict detection, the parent generation of the population is composed of two chromosomes extracted by the crossover operator. The start and end positions of one of the parent generation segments are randomly selected, and the genes of the offspring crossover segment are the same as it; in the other parent generation, the crossover segment is removed, and the other genes are added to the offspring in the original order to obtain a new offspring;

[0218] Mutation operations include mutation and inversion, which randomly extract the start and end positions of chromosomes and rearrange the mutated fragments.

[0219] In step S5, multi-agent reinforcement learning is used to plan the local trajectories and data acquisition of multiple drones, specifically:

[0220] The multi-UAV sensing and computation problem is modeled as a partially observable Markov decision process, defined as a tuple in, is the system state space, is the action space, is the state transfer function, is the system observation space, is a state-action pair, r is a reward function, γ is a discount factor, Ω is an observation function,

[0221] Each agent obtains a Observed According to their strategy u (a t |j t )Select an action As a result of the action, the agent will receive a reward and the next observation from the environment; the observation function Each agent has an action observation history Then their strategy is All agents share the same reward function The discount factor for future returns is γ∈[0,1), then the discounted feedback benefit is The joint strategy of the agents produces a value function, which is the expectation of discounted feedback, expressed as And the action value function The advantage function is F π (s t ,a t )=Q π (s t ,a t )-V π (s t );

[0222] The state space, observation space, action space and reward function are specifically defined as follows:

[0223] State Space S k is a collection of time-varying task information, and the state of each cluster head node is represented by a tuple express, represents the location coordinates of the cluster head node, C u represents the CPU cycles required to complete the data acquisition task calculation, D u It indicates the total amount of data that the UAV needs to obtain from each cluster during one mission cycle; S n It is used as a D2D communication drone; S u is the state of the drone performing the data acquisition task;

[0224] Observation space: The observation space of the agent is a subset of the state space. is the set K′ (K′≤K) of the most recent mission states within the observation range of the UAV; The set U′ (U′≤U) representing the most recent UAV state;

[0225] Action space: Based on the strategy and the observed system state, each UAV performs a joint action including data acquisition, transmission and calculation in each time slot; action space d u,kis the flight distance of the drone, and ω u are the pitch angle and yaw angle of the drone, η u and N u are respectively the task ratio and selection sequence number of the UAV for D2D communication, η u and N u This constitutes the computational strategy of UAV u;

[0226] Reward function: For each state, the agent chooses their action and receives a reward from the environment, converting the state-action pair As input function; the reward of the system

[0227] in, is the gain obtained by UAV u in acquiring data at time slot t, represents the total cost of time and energy consumption of UAV u, for punishment;

[0228]

[0229] Among them, χ u,k ∈{0,1} indicates whether the data of cluster head node k is first obtained by drone u; if so, then χ u,k =1, otherwise χ u,k =0; g(D u ,C u ) is the amount of data D obtained by UAV u u Instead of calculating the energy C u Related gain function; penalty factors are set for the two situations where the drone flies out of the target area and does not obtain enough data It is expressed as:

[0230]

[0231] in, Indicates that the drone u flies out of the target area, otherwise τ fly is the penalty value for the drone flying out of the area; Indicates the amount of data not acquired in the time slot, τ ina The penalty factor indicating the amount of data not obtained;

[0232] It is expressed as the sum of time and energy consumption, namely:

[0233]

[0234] in, and Indicates the weight of prioritizing time or energy consumption, It represents the gain obtained by UAV u when acquiring data at time t.

[0235] In step S6, the COMA algorithm is used to formulate the UAV flight strategy to adjust the flight angle, trajectory and calculation method of the UAV in real time, and complete the multi-UAV trajectory planning, data transmission and calculation tasks, specifically:

[0236] A centralized critic is used to calculate an agent-specific advantage function to solve the confidence allocation problem in multi-agent collaborative systems. The Q(s t ,a t ) is estimated in the concentrated state s t The conditional action a t Q value;

[0237] For each agent u, an advantage function can be calculated to convert the current action Q-value and marginalization , while keeping the actions of other agents a -u Fixed, expressed as follows:

[0238]

[0239] Among them, -u represents the agents other than agent u, F u (s t ,a t ) Compute a separate baseline for each agent that uses a centralized critic to reason about counterfactuals, where only the action of u changes, learning directly from the agent’s experience rather than relying on additional simulations, designed default actions, or reward models;

[0240] To optimize π u (·|j u ), use the policy gradient theorem, and minimize:

[0241]

[0242] In order to verify the effectiveness of the method of the present invention, a 3km×3km×1km 3D space is set, in which three types of sensor devices that generate video streams, audio streams or data streams are randomly distributed; the sensor will perform two clustering algorithms, and the amount of data in the cluster after the balanced data volume clustering algorithm is about 400MB. The data acquisition rate of each drone is 5MB / s, the unit energy cost is 0.9J, the CPU frequency range is [2,3]GHz, the energy consumption of the drone per CPU cycle is [1.5*10-10,2*10-10]J / circle, the flight speed range of the drone is [10,30]m / s, and the drone D2D communication range is [20,50]m;

[0243] like Figure 4 As shown in the figure, the method of the present invention compares the amount of data in each cluster of the traditional k-means clustering algorithm and the balanced data clustering algorithm; the figure clearly shows the imbalance of the amount of data obtained by the traditional clustering algorithm. For example, in the 10th and 15th clusters, the amount of data to be obtained is 800MB, while in clusters numbered 2, 3, and 4, the amount of data is less than 200MB. After using the balanced data clustering algorithm, the amount of data in each cluster is around 400MB, thereby more effectively balancing the time for each drone to obtain data.

[0244] like Figure 5 , 6 The performance comparison of the genetic algorithm, COMA algorithm and the algorithm proposed in this paper on the data acquisition and transmission calculation problem is shown in Figure 2. From the perspective of overall running time, the genetic algorithm relies on its powerful global search ability and adaptive processing ability to find a better solution in a relatively short time; while the COMA algorithm requires a large number of iterative training and strategy updates, consuming more computing resources and time. From the perspective of data acquisition, the genetic algorithm often abandons the poor data acquisition position in order to converge quickly, so that the amount of data in the environment cannot be fully acquired; the COMA algorithm can achieve collaboration between agents, share experience and learn from each other, and continuously update the strategy parameters of the agent during the training process. Each agent can optimize the solution to the problem as a whole. The algorithm proposed in this paper combines the advantages of the above two algorithms. While quickly converging on the global trajectory, it can also reduce the risk of falling into the local optimal solution. Through cooperation and learning between agents, the amount of data acquired is maximized within the flight time, and the resources in the environment are efficiently utilized to achieve the data acquisition and transmission calculation tasks of multiple drones.

[0245] like Figure 7As shown in the figure, the impact of different computing strategies on computing time is demonstrated. Under the premise of fixing the number of CPU cycles required to calculate the amount of data, the time to complete the data calculation will increase as the amount of data increases. If the drone only relies on local computing, when the amount of data is large, the drone needs to spend more time to process the data, which will have an adverse effect on the drone's load. However, if this article chooses to adopt a combination of D2D computing and local computing, when the amount of data is large, part of the data can be transmitted to nearby drones with smaller computing data via D2D, thereby greatly reducing the data calculation time. In addition, compared with the random data calculation strategy, this method can also better reduce the calculation time.

[0246] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A multi-UAV trajectory planning and data acquisition method based on the multi-traveling salesman problem, characterized by: The method comprises the following steps: Step S1: multiple sensors and cluster head nodes are arranged in the scene, the sensors transmit data to the cluster head node for storage, and the drone obtains the cluster head node data; Step S2: Establish a problem model, taking into account the clustering of sensors and the location of cluster head nodes, the set of service clusters of each drone, and the order of visiting these location nodes; Step S3: transform the problem model into a multiple traveling salesman problem; Step S4: using an improved genetic algorithm to perform global planning of multiple UAV trajectories; Step S5: Using multi-agent reinforcement learning to plan the local trajectories and data acquisition of multiple drones; Step S6: Use the COMA algorithm to formulate the UAV flight strategy to adjust the UAV's flight angle, trajectory and calculation method in real time to complete the multi-UAV trajectory planning, data transmission and calculation tasks.

2. The multi-UAV trajectory planning and data acquisition method based on the multi-traveling salesman problem according to claim 1 is characterized in that: In step S1, multiple sensors and cluster head nodes are arranged in the scene, the sensors transmit data to the cluster head node for storage, and the drone obtains the cluster head node data, specifically: Se sensors are randomly distributed in the scene. Each sensor collects information about the ground environment around it. The types of data collected by the sensor include humidity, temperature, noise, smoke concentration, and video data; K cluster head nodes are deployed in the scene. K cluster head nodes pre-collect the sensors se within their collection range and wait for the drones to perform data acquisition tasks, satisfying K>>U, where U is the total number of drones; Assume that the transmission power of the cluster head node is P k , the transmission power of other nodes in the cluster is P se , P k >>P se ; All sensor nodes are divided into K clusters. Each sensor must be bound to a cluster head node k. The received signal-to-noise ratio of cluster head node k is expressed as: Among them, d k,se is the distance between cluster head node k and sensor se; α is the path loss exponent, σ 2 is the noise power.

3. The multi-UAV trajectory planning and data acquisition method based on the multi-traveling salesman problem according to claim 1 is characterized in that: The problem model is established in step S2, specifically: Step S2-1: Establishing a kinematic model of the UAV to constrain the speed, angle and distance of the data acquired by the UAV during flight; Step S2-2: Establish a transmission channel model and analyze the average path loss of the transmission channel between the UAV and the cluster head node; Step S2-3: Establish a UAV data acquisition model to analyze the data acquisition rate, data acquisition time, and data acquisition energy consumption of the UAV during flight; Step S2-4: Establishing a UAV calculation model, analyzing the calculation mode of the UAV after acquiring data, and analyzing the time composition of the UAV in a data acquisition task cycle; Step S2-5: Establish a model for the problem of the amount of data that the drone needs to obtain from the cluster head node, optimize the clustering of the drone service, the total amount of data that the drone needs to obtain from each cluster during one mission cycle, the location of the cluster head node served by the drone, the order in which the drone visits these locations, the discretized location of the flight trajectory when the drone obtains data, and the communication calculation strategy adopted by the drone.

4. The multi-UAV trajectory planning and data acquisition method based on the multi-traveling salesman problem according to claim 3 is characterized by: In step S2-4, a drone computing model is established to analyze the computing mode of the drone after acquiring data, specifically: Assume that the server has a total computing resource C total , let C u represents the computing resources allocated to UAV u, then the UAV computing model is: in, Indicates the data acquisition time, Indicates the acquisition rate of the drone; when the drone completes a round of data acquisition tasks, the data acquisition tasks are placed in the local queue for calculation; Each drone is equipped with a CPU as a computing device. Due to its limited capabilities, it needs to transmit data to nearby drones through D2D for collaborative computing to achieve better performance; Establish a D2D communication model. The specific operations are as follows: Assume that the communication radius of the UAV is r, and the two UAVs can communicate only when the distance between them is within r. Using N u ={n|n∈U, d n,u <r} to represent the set of potential communicable UAVs of UAV u, the transmission rate between UAV u and UAV n is as follows: in, is the channel bandwidth allocated by the drone for D2D communication, is the transmission power from UAV u to UAV n, is the channel gain between UAV u and UAV n, d n,u represents the distance between UAV u and UAV n; The transmission time is: Among them, η u represents the proportion of computing tasks in D2D communication, η u ∈[0,1]; when the D2D communication mode is selected, the energy consumption of data transmission is: The calculation time is: in, represents the local CPU frequency of drone n; the total transmission time calculated by D2D communication is The energy consumed is: Among them, δ n (f n ) 2 is the energy consumption per CPU cycle of drone n; the total transmission energy consumption calculated by D2D communication is Therefore, the weighted sum of time and energy consumption when drone u chooses D2D communication mode is: Among them, θ t and θ e Respectively represent the time weighting coefficient and energy consumption weighting coefficient, satisfying θ t +θ e =1; Assuming that drone u chooses local computing, the time is: in, represents the local CPU frequency of drone u, (1-η u ) is the proportion of local computing data acquisition tasks; Energy consumption is: Among them, δ u (f u ) 2 is the energy consumption per unit CPU cycle of drone u; the weighted sum of time and energy consumption is:

5. The multi-UAV trajectory planning and data acquisition method based on the multi-traveling salesman problem according to claim 1 is characterized in that: In step S2-5, a model for the problem of the amount of data that the drone acquires from the cluster head node is established, specifically: in, is the transmission time, is the calculation time, T u It is the time it takes for the drone to complete a round of data acquisition tasks. The time it takes for the drone to acquire data during flight, the time it takes for the drone to transmit data between drones after acquiring the data, and the time it takes for the drone to perform local calculations and D2D communication calculations. The time it takes for the drone to complete a round of data acquisition tasks does not exceed the flight time it takes for the drone to traverse the cluster head nodes. in, The data acquisition time between UAV u and UAV n; is the data transmission time between UAV u and UAV n; The data calculation time between UAV u and UAV n when the D2D communication mode is selected; Calculate time locally for the drone; The flight time of the drone.

6. The multi-UAV trajectory planning and data acquisition method based on the multi-traveling salesman problem according to claim 5 is characterized in that: In step S3, the problem model is transformed into a multiple traveling salesman problem, specifically: From the expression T(u), we know that we need to optimize the clustering of sensors, the location of cluster head nodes, the set of service clusters for each drone, and the order of visiting these location nodes to minimize the total working time of the drone and balance the service time of each drone. The optimization problem is expressed as follows: C1: C2: C3: C4: C5: C6: C7:{maxT(u)-minT(u)}≤ε; C8: C9: C10: C11: C12: Among them, M u The set of clusters serving the u-th drone, Indicates the total amount of data that the drone needs to obtain from each cluster during one mission cycle; The location of the cluster head node serving the drone, is the set of discretized positions of the flight trajectory when the drone acquires data; u is drone u, and n is drone n; Indicates that the drone chooses local computing. Indicates that D2D transmission is used to perform calculations on drones. Each drone can only choose one calculation strategy, that is, γ k is the received signal-to-noise ratio of cluster head node k; is the threshold of the receiving signal-to-noise ratio; P k is the transmission power of the cluster head node; ave (dB) is the average path loss; Energy consumption for drone flight; Communication bandwidth for UAVs to obtain cluster head node data; The channel bandwidth allocated to the drone for D2D communication; d min The minimum safe distance between a drone and an obstacle and between two drones; is the center coordinate of the ob-th obstacle, P th is the threshold value; is the starting position of UAV u; B u is the bandwidth resource allocated to UAV u; B max is the maximum value of bandwidth resources; Constraint C1 means that the received signal-to-noise ratio of the cluster head node must be greater than the threshold, and constraint C2 means that the received power of the drone during flight must be greater than the receiving threshold; constraint C3 means that the elements within the cluster do not intersect, and C4 means that the cluster sets served by each drone do not intersect either; constraint C5 means that all drones must return to the starting position for charging after acquiring data; constraint C6 means that the time it takes for a drone to complete a round of data acquisition tasks does not exceed the flight time of the drone traversing the cluster head nodes; constraint C7 means that after a cycle, the difference between the longest working time and the shortest working time of the drone must be less than the threshold ε; constraint C8 means that each drone can only choose one computing strategy, and constraint C9 means that the energy consumption of the drone to complete a round of data acquisition tasks must be less than the drone's own power; constraint C10 is a bandwidth constraint for drones used for data acquisition and D2D communication; constraint C11 is to ensure the feasibility of drone flight, and constraint C12 is to ensure the safety of drone flight.

7. The multi-UAV trajectory planning and data acquisition method based on the multi-traveling salesman problem according to claim 1 is characterized in that: In step S4, the improved genetic algorithm is used to perform global planning of multiple UAV trajectories, specifically: According to the number of drones, the MTSP problem is decomposed into independent TSP sub-problems, and the independent TSP sub-problems are solved by improving the genetic algorithm through 2-opt optimization. Specifically: Step S4-1: Initialize the population; The population is binary-encoded, that is, the genotype of each individual is a binary-encoded symbol string; Step S4-2: Introduce the 2-opt optimization operator to optimize the inferior individuals separately before executing each genetic operation; In order to maintain the diversity of the population, new individuals will be randomly generated by the remaining Doping operators and directly added to the next generation population. Step S4-3: Calculate the fitness function; The inverse of the obtained drone task completion time function is used as a function to evaluate the individual fitness. The shorter the time, the higher the fitness, and the longer the time, the lower the fitness. Assume that the total number of individuals in the population is N, and the fitness of drone u is Among them, T(u) represents the time it takes for a drone to complete a round of data acquisition tasks; the probability of drone u being selected is Step S4-4: Determine whether the termination condition is met, if so, execute step S5, if not, execute step S4-5; Step S4-5: For the population that does not meet the termination condition, perform selection operation, crossover operation and mutation operation, and return to step S4-2 after the operation until the termination condition is met.

8. The multi-UAV trajectory planning and data acquisition method based on the multi-traveling salesman problem according to claim 1 is characterized by: In step S5, multi-agent reinforcement learning is used to plan the local trajectories and data acquisition of multiple drones, specifically: The multi-UAV sensing and computation problem is modeled as a partially observable Markov decision process, defined as a tuple in, is the system state space, is the action space, is the state transfer function, is the system observation space, is a state-action pair, r is a reward function, γ is a discount factor, Ω is an observation function, Each agent obtains a Observed According to their strategy u (a t |j t )Select an action As a result of the action, the agent will receive a reward and the next observation from the environment; the observation function Each agent has an action observation history Then their strategy is All agents share the same reward function The discount factor for future returns is γ∈[0,1), then the discounted feedback benefit is The joint strategy of the agents produces a value function, which is the expectation of discounted feedback, expressed as And the action value function The advantage function is F π (s t ,a t )=Q π (s,a t )-V π (s t ); The state space, observation space, action space and reward function are specifically defined as follows: State Space S k is a collection of time-varying task information, and the state of each cluster head node is represented by a tuple express, represents the location coordinates of the cluster head node, C u represents the CPU cycles required to complete the data acquisition task calculation, D u It indicates the total amount of data that the UAV needs to obtain from each cluster during one mission cycle; S n It is used as a D2D communication drone; S u is the state of the drone performing the data acquisition task; Observation space: The observation space of the agent is a subset of the state space. is the set K′ (K′≤K) of the most recent mission states within the observation range of the UAV; The set U′ (U′≤U) representing the most recent UAV state; Action space: Based on the strategy and the observed system state, each UAV performs a joint action including data acquisition, transmission and calculation in each time slot; action space d u,k is the flight distance of the drone, and ω u are the pitch angle and yaw angle of the drone, η u and N u are respectively the task ratio and selection sequence number of the UAV for D2D communication, η u and N u This constitutes the computational strategy of UAV u; Reward function: For each state, the agent chooses their action and receives a reward from the environment, converting the state-action pair " as input function; the reward of the system in, is the gain obtained by UAV u in acquiring data at time slot t, represents the total cost of time and energy consumption of UAV u, for punishment; Among them, x u,k ∈{0,1} indicates whether the data of cluster head node k is first obtained by drone u; if so, then x u,k =1, otherwise x u,k =0; g(D u ,C u ) is the amount of data D obtained by UAV u u Instead of calculating the energy C u Related gain function; penalty factors are set for the two situations where the drone flies out of the target area and does not obtain enough data It is expressed as: in, Indicates that the drone u flies out of the target area, otherwise τ fly is the penalty value for the drone flying out of the area; Indicates the amount of data not acquired in the time slot, τ ina The penalty factor indicating the amount of data not obtained; It is expressed as the sum of time and energy consumption, namely: in, and Indicates the weight of prioritizing time or energy consumption, It represents the gain obtained by UAV u when acquiring data at time t.

9. The multi-UAV trajectory planning and data acquisition method based on the multi-traveling salesman problem according to claim 1 is characterized in that: In step S6, the COMA algorithm is used to formulate the UAV flight strategy to adjust the flight angle, trajectory and calculation method of the UAV in real time, and complete the multi-UAV trajectory planning, data transmission and calculation tasks, specifically: A centralized critic is used to calculate an agent-specific advantage function to solve the confidence allocation problem in multi-agent collaborative systems. The Q(s t ,a t ) is estimated in the concentrated state s t The conditional action a t Q value; For each agent u, an advantage function can be calculated to convert the current action Q-value and marginalization , while keeping the actions of other agents a -u Fixed, expressed as follows: Among them, -u represents the agents other than agent u, F u (s t ,a t ) Compute a separate baseline for each agent that uses a centralized critic to reason about counterfactuals, where only the action of u changes, learning directly from the agent’s experience rather than relying on additional simulations, designed default actions, or reward models; To optimize π u (·|j u ), use the policy gradient theorem, and minimize:

Images