A collaborative perception method for UAV swarms that considers information age

Abstract

This invention relates to a collaborative perception method for UAV swarms that considers information age. Firstly, it introduces mobile edge computing into swarm intelligence perception, enabling UAVs to decide whether to perform local computation or directly transmit data back based on channel conditions when transmitting data. The invention also discloses a UAV trajectory and computation scheduling strategy based on reinforcement learning. First, the attributes of all points of interest are obtained and clustered according to these attributes. Then, based on the clustering results, the cluster centroids are expanded and redistributed using the Hungarian algorithm. UAVs select clusters, and reinforcement learning is used to select points of interest and transmission strategies. Through this approach, the invention solves the problem of poor information freshness in UAV-collected data in existing technologies.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of mobile crowd sensing, and particularly relates to a method for cooperative sensing of a UAV cluster considering information age. BACKGROUND

[0002] Mobile crowd sensing is becoming an attractive paradigm for large-scale data collection to support various applications, such as environmental monitoring, traffic flow estimation, and the like. Until recently, a mobile agent-based system has emerged. Unlike traditional human-centric mobile crowd sensing using smartphones and wearable devices, this new sensing mode utilizes the high agility and controllable mobility of mobile agents (such as UAVs, unmanned vehicles) to provide low-cost and high-quality data services, especially in some difficult-to-reach dangerous areas, such as forest floods, earthquakes, floods, and the like.

[0003] Information age (Aoi) is used to represent the freshness of collected data, and the information age is measured in the form of the elapsed time from the time of the latest received data to the perception data. Aoi is essentially different from the existing delay indicators or waiting time, and is a new time metric that can represent the freshness of information.

[0004] Reinforcement learning utilizes an agent to learn by trial and error in continuous interaction with the environment, and the agent learns the best strategy according to the environment feedback in each step of interaction with the environment. When the agent operates the environment, the environment changes, and the agent decides the strategy according to the new environment, so that the cumulative return obtained by the agent from the environment is maximized.

[0005] Existing works often ignore the limited battery life of UAVs when collecting data, and the freshness of the collected data. In addition, many traditional schemes also ignore the excellent computing performance of UAVs, and the use of local processing of UAVs can greatly reduce the transmission time and improve the freshness of information. SUMMARY

[0006] Technical problems to be solved

[0007] In order to avoid the shortcomings of the prior art, the application provides a method for cooperative sensing of a UAV cluster considering information age.

[0008] Technical scheme

[0009] A method for cooperative sensing of a UAV cluster considering information age, characterized in that the steps are as follows:

[0010] Step 1: constructing a UAV-aided UAV base station sensing system;

[0011] Step 2: constructing a perception model, a channel model, a calculation model, a point of interest information age model and an energy model of the unmanned aerial vehicle;

[0012] Step 3: establishing an optimization target according to the perception system and the related models;

[0013] Step 4: classifying points of interest (POI) by using a K-means algorithm improved based on a Hungarian algorithm;

[0014] Step 5: converting the behavior of the unmanned aerial vehicle to each cluster into a Markov problem;

[0015] Step 6: introducing a deep reinforcement learning algorithm to train the unmanned aerial vehicle to find an optimal collection point and a transmission calculation strategy.

[0016] A further technical solution of the application: the unmanned aerial vehicle assisted base station information perception system in step 1 comprises multiple unmanned aerial vehicles, multiple points of interest and a charging and base station; the unmanned aerial vehicles go to the multiple points of interest to perceive data, and return the data to the base station or perform local calculation after perceiving the data; when the power of the unmanned aerial vehicles is consumed, the unmanned aerial vehicles return to the base station to be charged; the unmanned aerial vehicles are configured in a hovering mode during data collection and data return; each point of interest has a threshold of information freshness.

[0017] A further technical solution of the application: the perception model in step 2:

[0018]

[0019] wherein ψ represents a probability of successful perception, loc(u j ,t) represents the position of the unmanned aerial vehicle j, and loc(p i ) represents the position of the point of interest j;

[0020] The channel model: the channel of the unmanned aerial vehicle is divided into K equal subcarriers, W B represents the bandwidth of the subcarrier, and the transmission rate of the unmanned aerial vehicle to the base station for data transmission can be represented as

[0021] R(x)=k*W B *log2(1+γ(x))

[0022] wherein k represents the number of subcarriers of the wireless channel allocated to the UAV in transmission at this moment, and γ(t) is the signal-to-noise ratio of the data transmission of the unmanned aerial vehicle to the base station, which can be represented as:

[0023]

[0024] wherein P represents the energy received by the base station from the unmanned aerial vehicle, and N0 represents the size of the noise at the receiving end of the base station; P TP(x) represents the communication transmission energy of the UAV La (x) represents the average path loss, which can be divided into visual distance communication Los and non-visual distance communication NLos, then P La (x) can be represented as:

[0025] PL a (x) = Pr Los (x) * PL Los (x) + Pr NLos (x) * PL NLos (x)

[0026] where Pr L (t) is the communication probability Pr LoS (x) can be represented as:

[0027]

[0028] where p0=233.98log 10 (h)0.95,r c = max{294.05log 10 (h) - 432.94, 18} and the path loss PL LoS (x) and PL NLoS (x) can be represented as:

[0029] PL LoS (x) = 30.9 + (22.5 - 0.5log 10 (h))·log 10 (||x-x b ||2) + 20log 10 (f c )

[0030] PL +LoS (x) = 32.4 + (43.2 - 7.6log 10 (h))·log 10 (||x-x b ||2) + 20log 10 (f c )

[0031] where fc c represents the carrier frequency, ||x-x b ||2 represents the distance of the UAV from the base station, x is the position of the UAV, and x b is the position of the base station;

[0032] The calculation model: for the calculation model of the unmanned aerial vehicle, considering the time of calculation and the energy of calculation, for the time of calculation can be expressed as:

[0033]

[0034] Wherein η represents the CPU cycle required for processing each bit of data, f represents the CPU frequency; D t The size of the calculation data is calculated;

[0035] The energy of calculation can be expressed as:

[0036] θ t = ψf 2 ηD t

[0037] Wherein ψ is a chip architecture related parameter;

[0038] The interest point information age model:

[0039]

[0040] Wherein τ t,p The information age τ t,p Of the interest point p at t time is represented as t-u(p), wherein u(p) represents the time of the last successful perception of the interest point p; The threshold value of the information age of each interest point POI is depicted, and the requirement of different places for the information age threshold value is depicted;

[0041] The energy model:

[0042]

[0043] Wherein W represents the weight of the unmanned aerial vehicle, ρ represents the density of air, A represents the total area of the unmanned aerial vehicle rotor, v t The speed of the unmanned aerial vehicle.

[0044] Further technical solutions of the present application: the optimization target in step 3 is described:

[0045]

[0046]

[0047] Wherein π n Represents the interest point selection and transmission calculation strategy of n unmanned aerial vehicles, N c Is the time duration considered, P represents the set of interest points POI, It is a new index based on time freshness; This indicates that the drone's energy must meet certain limits; if the drone's battery level is too low... min The drone will then need to return to its charging station.

[0048] A further technical solution of the present invention: The improved K-means algorithm based on the Hungarian algorithm is as follows:

[0049] First, the K-means algorithm is used to obtain m clusters, where m is the number of drones. The cluster centers can then be represented as:

[0050] G = [g1, g2, ..., g m ]

[0051] Where g i For cluster i, expand the cluster centers into a new set.

[0052]

[0053] in n is the number of points of interest, which will be calculated. Each element g i and POIp j The Euclidean distance is taken as c i,j This yields a new distance matrix C:

[0054]

[0055] The n points of interest are redistributed using the Hungarian algorithm.

[0056] A computer system is characterized by comprising: one or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method described above.

[0057] A computer-readable storage medium is characterized by storing computer-executable instructions, which, when executed, are used to implement the above-described method.

[0058] Beneficial effects

[0059] This invention provides a collaborative perception method for UAV swarms that considers information age, used to perceive points of interest within a range and calculate environmental information in time-sensitive scenarios such as earthquakes and floods. Compared with existing methods, this invention has the following advantages:

[0060] 1) A new index for information freshness is proposed, taking into account the real emergency situations in different regions within the area;

[0061] 2) It makes full use of the drone's computing power and adaptively optimizes the drone's data transmission method based on the network bandwidth status in the area;

[0062] 3) It eliminates the assumption of unlimited drone battery power and takes into account the drone battery power, making it more in line with real-world conditions. Attached Figure Description

[0063] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0064] Figure 1 This is a scene diagram from the present invention;

[0065] Figure 2 This is a diagram of the solution in this invention. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0067] This invention provides a collaborative perception method for UAV swarms that considers information age, comprising the following steps:

[0068] Step 1: Construct a drone-assisted drone base station perception system;

[0069] Step 2: Construct the UAV's perception model, channel model, computation model, point of interest information age model, and energy model;

[0070] Step 3: Establish optimization objectives based on the perception system and relevant models;

[0071]

[0072] Where π i N represents the interest point selection and transmission calculation strategy for i drones. c This refers to the duration of time we are considering. P represents the set of Points of Interest (POIs). This is a new indicator we propose based on time freshness.

[0073] Step 4: Classify the points of interest (POIs) using the K-means algorithm, which is an improvement on the Hungarian algorithm.

[0074] Step 5: Transform the drone's behavior for each cluster into a Markov problem;

[0075] Step six: Introduce deep reinforcement learning algorithm to train the UAV to find the optimal collection point and transmission calculation strategy.

[0076] The first step is to build a UAV-assisted base station information perception system, which includes multiple UAVs, multiple points of interest, and a charging and base station. The UAVs go to multiple points of interest to perceive data, and after perceiving the data, they transmit the data back to the base station or local calculation. When the UAV's power is consumed, it returns to the base station for charging. The UAVs are configured in hover mode during data collection and data transmission. Each point of interest has a threshold for information freshness.

[0077] The second step is to perceive data at point j by UAV j, and the perception probability model is:

[0078]

[0079] where ψ represents the probability of successful perception, loc(u j (t) represents the location of UAV j, and loc(p i ) represents the location of point j.

[0080] For the communication model between UAV j and base station b, we divide the UAV's channel into K equal subcarriers. W B represents the bandwidth of the subcarrier. The transmission rate of the UAV transmitting data to the base station can be represented as

[0081] R(x) = k * W B *log2(1 + γ(x))

[0082] where k represents the average number of subcarriers allocated to the UAV currently transmitting, and γ(x) is the signal-to-noise ratio of the UAV's data transmission to the base station, which can be represented as:

[0083]

[0084] where P is the transmit power at the UAV end, and N0 represents the size of the noise at the base station receiver. PL a (t) represents the average path loss. Specifically, the average path loss PL a (t) can be represented in two cases, namely, line-of-sight (LoS) and non-line-of-sight (NLoS). Therefore, the average air-ground path loss from the UAV to the base station can be represented as:

[0085] PL a (x) = Pr LoS (x) * PL Los (x) + Pr NLoS(x) * PL NLos (x)

[0086] where Pr LoS (x), Pr NLoS (x) is the probability that there is a line-of-sight (LoS) communication and a non-line-of-sight (NLoS) communication between the UAV and the base station BS at x. PL Los (x) and PL NLos (x) represent the path loss under LoS and NLoS communication, respectively. Specifically, Pr LoS (x) can be calculated by the following equation:

[0087]

[0088] where p0= 233.98 log 10 (h)0.95, r c = max{294.05 log 10 (h) - 432.94, 18} where Pr NLoS (x) = 1 - Pr LoS (x). And the path loss PL LoS (x) and PL NLoS (x) can be represented as:

[0089] PL LoS (x) = 30.9 + (22.5 - 0.5 log 10 (h)) · log 10 (||x - x b ||2) + 20 log 10 (fc c )

[0090] PL NLo (x) = 32.4 + (43.2 - 7.6 log 10 (h)) · log 10 (||x - x b ||2) + 20 log 10 (fc c )

[0091] where f v represents the carrier frequency, ||x - x b ||2 represents the distance between the UAV and the base station, x is the position of the UAV, and x b is the position of the base station.

[0092] The information age is introduced to describe the freshness of the data perceived by the UAV at the base station. Initially, the information age of each point of interest is zero, and there is a threshold of information age τ threWhen the UAV flies to the point of interest, collects data, and then returns the data to the base station through local computing or wireless communication. Therefore, the information age consists of the transmission time of the UAV or the computing time of the UAV. We use τ t,p to represent the information age of the point of interest P at time t t,p = t - u(p), where u(p) represents the time of the last successful perception of the point of interest p, and we take the part exceeding the information age threshold as our indicator, denoted as

[0093]

[0094] For the allocated wireless channel that cannot support the transmission of the collected sensing data, the UAV will adopt edge computing for processing. We denote η as the number of CPU cycles required to process each bit of data, and f as the CPU frequency of the local processor. Therefore, the time of local computing of the UAV for data size D t can be represented as:

[0095]

[0096] Regarding the computing model of the UAV, we consider that after the UAV processes, the UAV can directly return the processed results to the base station without occupying the system transmission time.

[0097] The computing energy of the UAV can be represented as:

[0098] θ t = ψf 2 ηD t

[0099] where ψ is a chip architecture-related parameter.

[0100] For the energy model of the UAV, during the process of UAV perception and computing, we consider the energy consumption of the UAV flight and the energy consumption of the UAV edge computing, which is relatively small compared to the transmission energy consumption of the UAV, so we do not consider the transmission energy consumption of the UAV. For the computing energy consumption of the UAV, we have given it above.

[0101] The flight energy of the UAV can be represented as:

[0102]

[0103] where W represents the weight of the UAV, ρ represents the density of air, and A represents the total area of the UAV rotor. v t represents the speed of the UAV.

[0104] In step three, given our scenario, which includes multiple drones, multiple points of interest (POIs), a charging station, and a base station, a drone flies to a POI, uses a perception model to perceive the POI, and after perceiving the data, decides whether to use a channel model to transmit the data back to the base station or a local computing model to complete the computation on the drone. In addition, we utilize the computational energy model and flight energy model from step two. When the drone's battery is depleted, it returns to the base station to recharge. This leads to the following optimization objective.

[0105]

[0106]

[0107] Where π i N represents the interest point selection and transmission calculation strategy for i drones. c This refers to the duration of time we are considering. P represents the set of Points of Interest (POIs). This is the new indicator based on time freshness that we proposed above. This indicates that the drone's energy must meet certain limits; if the drone's battery level is too low... mip The drone then needs to return to its charging station. We implicitly referenced this model during the simulation for both the perception and computational aspects.

[0108] In step four, considering that drones may exhibit a "turnaround" phenomenon during flight, significantly consuming their energy, we consider a new K-means-based clustering algorithm. Traditional K-means algorithms, when faced with our problem scenario, exhibit outliers. The presence of these outliers can lead to task imbalances in subsequent drone missions.

[0109] To distribute interest points more evenly, we use the Hungarian algorithm to redistribute them. We first use the K-means algorithm to obtain m clusters, where m is the number of drones. The cluster centers can then be represented as:

[0110] G = [g1, g2, ..., g m ]

[0111] Where g i For cluster i, expand the cluster centers into a new set.

[0112]

[0113] in n is the number of points of interest, which will be calculated. Each element g i Points of Interest (POIs)j The Euclidean distance is taken as c i,j This yields a new distance matrix C:

[0114]

[0115] The n points of interest are redistributed using the Hungarian algorithm.

[0116] In step five, within each time slice, the drone may perform operations such as flight, hovering perception, and computation. In this stage, we transform our problem into a Markov Decision Process (MDP), and propose a reinforcement learning-based approach to solve it. First, we define MDP =<S,A,R,γ> , where S represents the set of states, A represents the corresponding decision, R represents the reward obtained by taking the decision, and γ is the discount factor for future rewards.

[0117] The state s∈S at time t is defined as follows:

[0118]

[0119] where t∈[1,N] c ] represents the current time in N c The sequence in N time slices, c is the length of the entire time slice. N and M represent the set of points of interest (POIs) to be sensed and the set of drone swarms, respectively. i Let e ​​represent the position coordinates of the point of interest (POI) i. j U represents the remaining energy of drone j. j Let τ represent the position of drone j at time t. i This represents the information age of the point of interest (POI) i at this moment. c represents the maximum information age limit for point of interest (POI) i. j The six states of UAV j can be categorized into decision-making, flight, perception, transmission, computation, and charging states. In the decision-making state, the UAV spends a time slice in place making a decision, selecting a Point of Interest (POI) to perceive and a data processing method. Then, in the flight state, it flies to the selected POI. In the perception state, it perceives information from the POI until successful data acquisition. Based on the decision, in the transmission state, it chooses to transmit the data to the base station, or in the computation state, it processes the data locally. If the UAV's energy falls below the energy threshold for that location, it enters the charging state, abandons the perception task, and automatically returns to the base station to recharge. After charging is complete, the UAV will return to the selected POI to perceive again.

[0120] In this, we denote the decision set of UAV j as

[0121]

[0122] where denote the transmission strategy that needs to be transmitted to the base station or local computation, respectively, where denotes the selected local computation, denotes the selected point of interest (POI) for data transmission. denotes which POI UAV j chooses to perform the task. If denotes the selected point of interest (POI) i to perform the task.

[0123] Rewards In a Markov decision environment, given a state s t When an agent makes a decision, it will get the corresponding reward. In this paper, in order to better depict the reward, we need to consider the AoI of the POI, the AoI deadline, the remaining energy of the UAV, the available bandwidth of the UAV, and the distance of the UAV.

[0124]

[0125] In the formula, denotes the AOI when POI j is selected, τ j denotes the AOI after the task is completed. d i,j denotes the distance between UAV i and PoI j. v denotes the speed of the UAV. a is a control parameter.

[0126] We denote the UAV j in the Markov decision as a mapping from state s to decision a. Formally, we denote it as π i ∶s→a i . So all the UAV strategies can be represented as π = {π1, π2,..., π m}, which shows how the UAV plans the path and allocates limited UAV communication resources to maximize the size of effective sensing data considering the remaining energy of the UAV, communication resources, and AoI deadline. Therefore, for each UAV i, the DDQN method is used to make decisions, and the decision-making process is as follows:

[0127] The state information of the agent is input, and the network outputs the value of each action of the agent, that is, the Q value. For DDQN, it is not directly in the target network to find the Q value of each action, but in the current network to find the action a corresponding to the maximum Q value,

[0128]

[0129] where s t+1 is the state of the agent at the next time step, and the action is obtained by the target network.

[0130]

[0131] where the decay factor 0 < γ < 1 represents the agent's concern for the present reward relative to future rewards. Then the computed value of the target Q value is

[0132]

[0133] We designed a two-stage decision-making process to make decisions. First, we want to cluster POIs by spatiotemporal information, and then assign each cluster to a UAV for execution. Then each UAV i selects a PoI and whether to process data locally or directly return data according to the cluster it is assigned to The best decision-making process can be calculated as follows: for a given state s t

[0134]

[0135] To describe the training process, we take N c time slices as a round, and in each training round, when the UAV completes the training task, the UAV stores the experience in his experience pool. After each round, the UAV extracts N b experiences, and the parameters ω i are updated by the gradient loss function.

[0136]

[0137] where ω i represents the parameters of the target network.

[0138] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any skilled person in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application.

Claims

1. A method for cooperative sensing of a UAV swarm considering information age, characterized in that The steps are as follows: Step 1: Constructing a UAV-assisted base station sensing system; the UAV-assisted base station information sensing system contains multiple UAVs, multiple points of interest, and a charging and base station; the UAVs go to multiple points of interest POI to sense data, and after sensing the data, the data is transmitted back to the base station or local calculation; when the power of the UAV is consumed, it returns to the base station for charging; the UAV departs from the base station position and is configured in a hovering mode during data collection and data transmission; each point of interest has a threshold for information freshness; Step 2: Constructing a UAV sensing model, channel model, calculation model, point of interest information age model, and energy model; the sensing model: wherein is expressed as a probability of success of perception, represents the position of the drone j, represents the position of the point of interest j; The channel model: the channel of the unmanned aerial vehicle is divided into K equal subcarriers, The transmission rate of the unmanned aerial vehicle to the base station can be represented as where k is expressed as the number of subcarriers of the wireless channel allocated on average to the UAVs that are in transmission at this moment, The signal-to-noise ratio for the UAV-to-base station data transmission can be expressed as: Where P is expressed as the energy received at the base station from the drone, expressed as the size of the noise at the base station receiver end; expressed as the average path loss, which can be divided into line-of-sight communication Los and non-line-of-sight communication NLos, then can be expressed as: wherein the probability of communication for Los in the field environment may be expressed as: wherein , and path loss and may be expressed as: wherein denotes a carrier frequency, denotes a distance of the drone from the base station, is a position of the drone, is a position of the base station; The calculation model: for the calculation model of the UAV, the time and energy of the calculation are considered, and the calculation time can be represented as: wherein CPU cycles required to process each bit of data, CPU frequency; size of the data being computed; The energy for the calculation can be expressed as: wherein is a chip architecture dependent parameter; The point of interest information age AOI model: wherein to represent a point of interest information age at time t wherein, to represent a point of interest time of the last successful perception; a threshold of information age of each point of interest POI, which depicts the requirement of information age threshold in different places; The energy model: wherein W represents the weight of the drone, p represents the density of air, A represents the total area of the drone rotor, V represents the speed of the drone; Step 3: Establishing an optimization target according to the sensing system and related models; the optimization target: wherein represent n Point of Interest selection and transmission computation strategy for a fleet of drones, is the time duration considered, P represents a set of Points of Interest, POI, is an index based on time freshness; indicates that the energy of the drone must satisfy a certain range, if the energy of the drone is small the drone needs to return to charge; Step 4: Using the improved K-means algorithm based on the Hungarian algorithm to classify the points of interest POI; the improved K-means algorithm based on the Hungarian algorithm is as follows: First, use the K-means algorithm to get m clusters, where m is the number of UAVs, and the cluster center can be represented as: wherein is the center of cluster i, extend the cluster center to a new set : wherein n is the number of points of interest will be calculated each element and the Euclidean distance of the POI as a new distance matrix C is obtained: Use the Hungarian algorithm to redistribute n points of interest; Step 5: The drone translates the behavior of each cluster into a Markov problem; first define where represents a set of states, represents the corresponding decision taken, represents the reward obtained by taking that decision, is the discount factor for future rewards; State is defined at time t as follows wherein represents the sequence of the current time in is the length of the entire time slice; N and M represent the set of points of interest POI and the set of UAV clusters that need to be perceived, respectively; represents the location coordinates of the point of interest POI i, represents the remaining energy of the UAV j, represents the location of the UAV j at the beginning of the time t; represents the information age of the point of interest i at this moment, represents the maximum information age limit of the point of interest i, represents the six states of the UAV j, which can be divided into decision state, flight state, perception state, transmission state, calculation state and charging state; the UAV spends one time slice in the decision state to make a decision in place, selects the point of interest to be perceived and the processing method of data, then flies to the selected point of interest in the flight state, perceives the information of the point of interest in the perception state until the information is successfully collected, selects to transmit the data to the base station in the transmission state according to the decision information, or performs local processing on the data in the calculation state; if the energy of the UAV is lower than the energy threshold of the location, the UAV will enter the charging state and give up the perception task, and automatically returns to the base station for charging; after the charging is completed, the UAV will continue to return to the selected POI for perception again;​ The decision set of UAV j can be represented as wherein , respectively indicate whether a transmission strategy needs to be communicated to the base station or computed locally, wherein indicates that a local computation is selected, indicates that a POI is selected for data transmission; indicates which POI the drone j selects to perform the task; if = i indicates that the POI i is selected to perform the task; Rewards In a Markov decision environment, given a state When an agent performs a decision, it will get the corresponding reward; in order to better depict the reward, the AOI of the POI, the residual energy of the AOI deadline UAV, the available bandwidth of the UAV and the distance of the UAV need to be considered; In the formula, denotes the AOI when the POI j is selected, denotes the AOI after the selection task is completed; denotes the distance between the UAV i and the POI j; denotes the speed of the UAV, is a control parameter; Step 6: Introducing a deep reinforcement learning algorithm to train the UAV to find the optimal collection point and transmission calculation strategy; The state information of the agent is input, and the network outputs the value of each action of the agent, that is, the Q value; for DDQN, the Q value of each action is not directly found in the target network, but the action a corresponding to the maximum Q value is found in the current network, wherein, is the state of the agent at the next time, and the action is obtained by the target network the value of Q corresponding to: where the decay coefficient represents the agent's concern for the present reward in the future relative to the recent rewards; then the computed value of the target Q value is given by A two-stage decision making process is designed for decision selection: first, POIs are clustered by spatio-temporal information, and then each cluster is assigned to a UAV for execution; then each UAV i makes a decision based on the cluster it is assigned to from which to select POIs and whether to process data locally or to directly backhaul data, and the final optimal decision selection process can be computed as follows for a given state To describe the training process, Each time slice is considered a round. Within each training round, after the drone completes its training task, the drone stores its experience in its experience pool. At the end of each round, the drone extracts... One experience, parameters Update using the gradient loss function; wherein represents a parameter of the target network.

2. A computer system, characterized by Comprise: One or more processors, a computer readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method of claim 1.

3. A computer-readable storage medium, characterized in that Computer executable instructions are stored, and the instructions are executed to implement the method of claim 1.

Images