Unmanned aerial vehicle assisted environment state sensing and data collecting method

Abstract

The invention relates to an unmanned aerial vehicle assisted environment state sensing and data collecting method, and belongs to the technical field of unmanned aerial vehicles. The method comprises the following steps: constructing an environment state perception and data collection scene assisted by an unmanned aerial vehicle, and modeling a system model; establishing an unmanned aerial vehicle channel model in an unmanned aerial vehicle assisted environment state perception and data collection scene, and modeling a data collection rate; modeling an interest point detection probability sum; modeling system information age; modeling the total energy consumption of the unmanned aerial vehicle; modeling unmanned aerial vehicle sensing and data collection limiting conditions; modeling a system state space, an action space and a reward function; modeling and training a multi-agent depth deterministic strategy gradient model combined with long and short term memory and an attention mechanism; and determining a flight path, perception scheduling and data collection strategy of the unmanned aerial vehicle based on the trained model. According to the method, the man-machine flight path, the perception scheduling and the data collection strategy are jointly optimized, and the award function maximization in the environment state perception and data collection scene assisted by the unmanned aerial vehicle is realized.

Description

Technical Field

[0001] The invention belongs to the technical field of unmanned aerial vehicles, and relates to an unmanned aerial vehicle-assisted environmental state perception and data collection method. Background Art

[0002] A drone is a powered aircraft controlled by remote control or computer program, which is reusable and does not require manning. Due to its advantages such as high maneuverability, low cost, strong concealment and easy deployment, drones are widely used in both military and civilian fields. In order to overcome the limitations of spectrum resources and hardware dependence, the perception and communication fusion (ISAC) technology can be used. By deploying communication and sensor modules on drones, drones can perform target perception while communicating with other users. In drone-assisted communication perception scenarios, how to comprehensively consider the diverse service requirements and complex network characteristics, and design reasonable and efficient communication perception scheduling and trajectory planning strategies are important issues that need to be solved urgently.

[0003] Existing studies have designed UAV communication perception scheduling and trajectory planning strategies. However, most existing studies focus on optimizing a single performance indicator, and rarely consider the joint optimization of target perception and data transmission performance. In addition, existing studies are mostly conducted on static scenarios, and rarely consider the long-term performance optimization of the system. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a drone-assisted environmental state perception and data collection method, which includes multiple drones, a ground base station and multiple ground points of interest (PoI), models the reward function related to the detection probability and system information age of the points of interest, and the total energy consumption of the drone as the optimization target, and determines the drone's flight strategy, perception scheduling and collection strategy.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A method for sensing and collecting environmental conditions assisted by a drone, the method comprising the following steps:

[0007] S1. Build UAV-assisted environmental state perception and data collection scenarios, and model the system model;

[0008] S2, establish the UAV channel model and model the data collection rate in the UAV-assisted environmental state perception and data collection scenario;

[0009] S3, modeling interest point detection probability and;

[0010] S4, modeling system information age;

[0011] S5, total energy consumption of modeled UAV;

[0012] S6, modeling drone perception and data collection constraints;

[0013] S7. Modeling system state space, action space and reward function

[0014] S8. Model and train a multi-agent deep deterministic policy gradient model combining long short-term memory and attention mechanism;

[0015] S9, determine the flight trajectory, perception scheduling and data collection strategy of the drone based on the trained model;

[0016] Further, in step S1, the drone-assisted environmental state perception and data collection scenario includes multiple drones, a ground base station, and multiple ground points of interest (PoIs); the PoIs collect environmental state information, and the drone detects the state of the PoIs, and if the detection is successful, collects the data of the PoIs;

[0017] Let U n represents the nth drone, 1≤n≤N; divide the system time into T time slots of equal length, with the time slot length being τ; let θ n,t represents the tth time slot U n The flight angle, θ n,t ∈[0,2π]; let represents the tth time slot U n The position of H is U n The flight altitude of the drone is expressed as follows: Where v is the flight speed of the drone;

[0018] make represents the position of the mth PoI, 1≤m≤M, M is the total number of PoIs; assuming that the PoIs in the scene transmit data to the drone through frequency division multiple access OFDMA technology, and the bandwidth of each subchannel is B.

[0019] Further, in step S2, the UAV channel model is defined as:

[0020] The channel between the UAV and the PoI is modeled as a probabilistic line-of-sight LOS channel. represents the tth time slot U n The probability that the channel between and the mth PoI is a LOS channel is expressed as:

[0021]

[0022] where a and a′ are constants related to the environment, θ n,m,t is the tth time slot U n The elevation angle between and the mth PoI is expressed as: dn,m,t is the tth time slot U n The distance between the mth PoI is expressed as:

[0023]

[0024] make is the tth time slot U n The path loss of the LOS channel between and the mth PoI is expressed as:

[0025]

[0026] where α F is a constant that depends on the channel frequency and the speed of light, α LOS is the average additional line-of-sight path loss from the environment;

[0027] make is the tth time slot U n The path loss of the NLOS channel between and the m-th PoI is expressed as:

[0028]

[0029] where α NLOS is the average additional non-line-of-sight path loss from the environment;

[0030] Let h n,m,t is the tth time slot U n The average path loss between and the mth PoI is expressed as:

[0031]

[0032] In step S2, the data collection rate is defined as:

[0033] Let γ n,m,t is the tth time slot U n The signal-to-noise ratio between the mth PoI and the mth PoI is expressed as:

[0034]

[0035] Where P m represents the transmission power of the mth PoI, and N0 represents the noise power spectral density at the UAV receiver;

[0036] Let R n,m,t is the tth time slot U n The data transmission rate between and the mth PoI is expressed as:

[0037] R n,m,t =Blog2(1+γ n,m,t ).

[0038] In step S3, the interest point detection probability and is defined as:

[0039] make For U n The corresponding received power when detecting the mth PoI in the tth time slot is expressed as:

[0040]

[0041] Where P s Indicates the transmission power used by the drone when detecting the target, G t represents the transmitting antenna gain of the UAV radar, G r represents the radar receiving antenna gain, σ m represents the radar cross-section of the mth PoI, λ represents the wavelength of the radar signal, L rt represents the loss coefficient;

[0042] make represents the tth time slot U n The detection probability corresponding to detecting the mth PoI is expressed as:

[0043]

[0044] where I0(·) represents the zero-order Bessel function, σ 2 Represents the noise power when the UAV perceives the target; V T Represents the signal amplitude threshold, let p fa Denotes the radar false alarm probability as a given constant, we can get:

[0045]

[0046] Let μ n,m,t ∈{0,1} is the PoI perception association variable, μ n,m,t =1 means U n The mth PoI is sensed in the tth time slot, and vice versa. n,m,t =0; by the end of the tth time slot, the mth PoI is at least n Probability of successful detection It is expressed as:

[0047]

[0048] The detection probability and p of all PoIs sum It is expressed as:

[0049]

[0050] Further, in step S4, the system information age is defined as:

[0051] Let β n,m,t ∈{0,1} is the data collection variable of PoI, β n,m,t =1 means U n Collect the data of the mth PoI at the tth time slot, otherwise β n,m,t =0;

[0052] make The mth PoI transmits data to U in the tth time slot n The duration of is expressed as:

[0053]

[0054] Where D m,t is the remaining data amount to be transmitted to the UAV by the mth PoI at the end of the tth time slot, which is expressed as:

[0055]

[0056] D m represents the initial data volume of the mth PoI;

[0057] Let A m,t is the information age of the mth PoI in the tth time slot, which is expressed as:

[0058] A m,t =(ts m ) +

[0059] Among them, (x) + =max{0,x},s m is the time slot where the mth PoI starts to transmit data, which is expressed as:

[0060]

[0061] Let A represent the total information age of the system, which is expressed as:

[0062]

[0063] Further, in step S5, the total energy consumption of the UAV is modeled as:

[0064] Let P f is the flight power of the UAV, which is expressed as:

[0065]

[0066] Among them, v tiprepresents the tip speed of the UAV rotor blade, v0 represents the average speed caused by the UAV rotor; constants c1, c2, c3 depend on the UAV power, rotor and air density;

[0067] Let p h is the hovering power of the drone per time slot, which is expressed as:

[0068] p h =c1+c2

[0069] make For UAV n Flight status variable indicator variable, Indicates U n In the tth time slot, it is in the flying state, otherwise it is in the hovering state.

[0070] Let E n,t For U n The energy consumption in the tth time slot is specifically expressed as:

[0071]

[0072] Let E n For UAV n The total energy consumption is specifically expressed as:

[0073]

[0074] Let E be the total energy consumption of the drone to collect data, which can be expressed as:

[0075]

[0076] Further, in step S6, the constraints on drone perception and data collection include UAV-PoI association constraints, perception constraints, data transmission constraints, and drone flight constraints, where:

[0077] 1) UAV-PoI association constraints include:

[0078]

[0079]

[0080] 2) Target perception constraints include:

[0081]

[0082] where r e is the effective detection radius;

[0083]

[0084] where p th is the detection probability threshold when starting data transmission;

[0085] 3) Data transmission restrictions include:

[0086]

[0087] 4) UAV trajectory restrictions include:

[0088] a) The flight position restrictions of the drone are:

[0089]

[0090] Among them, l th is the minimum safe distance between two drones;

[0091] b) The energy consumption limit of the drone is:

[0092]

[0093] in, Indicates drone U n The initial energy, Indicates U n The remaining energy threshold.

[0094] Further, in step S7, the system state space, action space and reward function are modeled, specifically including:

[0095] (1) Let S t ∈S represents the system state space of the tth time slot, which is modeled as:

[0096]

[0097] in Denotes the UAV U in the tth time slot n status, The end time slot U n The remaining energy can be expressed as:

[0098]

[0099] represents the state of the mth PoI in the tth time slot, p m,t is the detection probability of the mth PoI by the end of the tth time slot, which is expressed as:

[0100]

[0101] (2) Let A t ∈A represents the action space of the tth time slot, then: At ={a 1,t ,…,a n,t ,…,a N,t}, where a n,t Denotes the UAV U in the tth time slot n Action Space

[0102] (3) Let R t ∈R represents the reward function of the tth time slot, then R t = {r 1,t ,…,r n,t ,…,r N,t}, r n,t Denotes the UAV U in the tth time slot n In status n,t When selecting action a n,t The rewards obtained are specifically modeled as follows:

[0103]

[0104] Among them, ω1, ω2, ω3, ω4 represent weight factors, Denotes the UAV U in the tth time slot n Data collection rewards,

[0105]

[0106] Denotes the UAV U in the tth time slot n Perceived Rewards:

[0107]

[0108] Denotes the UAV U in the tth time slot n The associated data collection task information age penalty items are:

[0109]

[0110] C1 and C2 are positive numbers, representing the penalty for the drone returning to the starting point without collecting all the data, and the penalty for not returning to the starting point when the remaining battery power of the drone is below the threshold.

[0111] Further, in step S8, a multi-agent deep deterministic policy gradient (MADDPG) combining long short-term memory (LSTM) and attention mechanism is modeled and trained, specifically including:

[0112] (1) Constructing a MADDPG network based on LSTM and attention mechanism: The constructed network includes a real actor network with parameter ω, a real critic network with parameter ω′, a target actor network with parameter θ, and a target critic network with parameter θ′; the fully connected layer FCN in the four networks consists of a cascaded LSTM layer and a soft attention layer;

[0113] In the cascaded LSTM layer, let f t represents the forget gate of the tth time slot, c t represents the cell state of the tth time slot, h t represents the hidden state of the tth time slot, which is specifically expressed as:

[0114] f t =σ l (W f ·[x t +h t-1 ]+b f )

[0115] c t =tanh(W c ·[x t +h t-1 ]+b c )*i t +f t *c t-1 h t =o t *tanh(c t )

[0116] Among them, i t represents the input gate of the tth time slot; o t represents the output gate of the tth time slot; W f , W c , b f , b c represents network parameters; h t-1 The hidden state of the t-1th time slot; c t-1 represents the cell state of the t-1th time slot; * represents the Hadamard product of the matrix; σ l Represents the sigmoid activation function;

[0117] In the soft attention layer, let Indicates the tth time slot and time slot The attention weight between is expressed as:

[0118]

[0119] in, represents the tth time slot ht and c t The state summary vector u t The transpose of

[0120] Let h′ t represents the updated hidden state of the tth time slot, which is expressed as:

[0121]

[0122] (2) Initialize network parameters: The current Actor network interacts with the environment, inputs state S to the real Actor network to obtain action A, applies action A to the environment, switches to state S′ at the next moment, and obtains reward R; stores the four-tuple (s, a, r, s′) in the experience pool;

[0123] (3) Sampling and parameter training: Take samples from the experience pool for training and transform the state S of the tth time slot t and action A t Input into the real Critic network to calculate the current Q(S t ,A t ; θ) value; the state S at the next moment t+1 Input to the target Actor network to get action A t+1 ; Change state S t+1 and action A t+1 Input into the target Critic network to calculate the target value; order Represents the loss function between two Q values. By minimizing the loss function, the parameter ω′ of the real critic network is updated.

[0124] (4) Update the actual Actor network parameters ω based on the policy gradient method:

[0125]

[0126] Where J is the agent's objective function, represents the gradient of f(x) with respect to x; μ(S t ,A t ; ω) represents the strategy of mapping the real Actor network from state to action; α is the learning rate of the gradient descent algorithm, 0≤α≤1;

[0127] (5) Update the network parameters based on the soft update method: Use the parameters ω of the actual actor network and ω′ of the actual critic network to update the parameters θ of the target actor network and θ′ of the target critic network, which can be expressed as:

[0128] ω′←ρω+(1-ρ)ω′

[0129] θ′←ρθ+(1-ρ)θ′

[0130] Where ρ represents the learning rate, ρ∈(0,1) and ρ<<1;

[0131] (6) Repeat the training process: first determine whether the termination condition has been reached. If not, return to step (2) and use the real Actor network to sample, and repeat this cycle until the termination condition is met. After the training is completed, the result obtained by the algorithm is the real Actor network. In the actual decision-making, when a state is input, the Actor network will output an optimal action. The other three networks are the optimal auxiliary networks obtained through training.

[0132] Furthermore, in step S9, the trained MADDPG network combining LSTM and attention mechanism is used to determine the flight strategy, perception scheduling and data collection strategy of the drone with the goal of optimizing the reward function while satisfying the drone perception and data collection constraints.

[0133] The beneficial effects of the present invention are:

[0134] The present invention maximizes the reward function related to the detection probability of points of interest, the system information age, and the total energy consumption of drones by jointly optimizing the flight strategy, perception scheduling, and collection strategy of drones.

[0135] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0136] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0137] Figure 1 A schematic diagram of a drone-assisted environmental state perception and data collection scenario of the present invention;

[0138] Figure 2 A flow chart of the UAV flight strategy, perception scheduling and collection strategy design method of the present invention. DETAILED DESCRIPTION

[0139] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0140] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0141] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0142] See also Figure 1-2 , which is a UAV-assisted environmental state perception and data collection design method.

[0143] like Figure 1 The UAV-assisted environmental state perception and data collection scenario diagram constructed by the present invention shown in the figure, in which there are UAVs, a ground base station and multiple ground points of interest (PoIs), by jointly designing the flight strategy, perception scheduling and collection strategy of the UAV, the reward function related to the detection probability and system information age of the points of interest, and the total energy consumption of the UAV can be maximized. The UAV-assisted environmental state perception and data collection scenario consists of multiple UAVs, a ground base station and multiple ground points of interest; PoIs collect environmental state information, and the UAV detects the state of the PoIs. If the detection is successful, the data of the PoIs is collected;

[0144] like Figure 2 The flowchart of the method for designing the UAV flight strategy, sensing scheduling and collection strategy of the present invention is shown, which includes the following steps:

[0145] S1, modeling system model;

[0146] S2, establish the UAV channel model and model the data collection rate in the UAV-assisted environmental state perception and data collection scenario;

[0147] S3, modeling interest point detection probability and;

[0148] S4, modeling system information age;

[0149] S5, total energy consumption of modeled UAV;

[0150] S6, modeling drone perception and data collection constraints;

[0151] S7. Modeling system state space, action space and reward function

[0152] S8. Model and train a multi-agent deep deterministic policy gradient model combining long short-term memory and attention mechanism;

[0153] S9, determine the flight trajectory, perception scheduling and data collection strategy of the drone based on the trained model;

[0154] Step 1: Model the system.

[0155] Let U n represents the nth drone, 1≤n≤N; divide the system time into T time slots of equal length, with the time slot length being τ; let θ n,t represents the tth time slot U n The flight angle, θ n,t ∈[0,2π]; let represents the tth time slot U n The position of H is U n The flight altitude of the drone is expressed as follows: Where v is the flight speed of the drone;

[0156] make represents the position of the mth PoI, 1≤m≤M, M is the total number of PoIs; assuming that the PoIs in the scene transmit data to the drone through frequency division multiple access OFDMA technology, and the bandwidth of each subchannel is B.

[0157] Step 2: Establish the UAV channel model.

[0158] The channel between the UAV and the PoI is modeled as a probabilistic line-of-sight LOS channel. represents the tth time slot U n The probability that the channel between and the mth PoI is a LOS channel is expressed as:

[0159]

[0160] where a and a′ are constants related to the environment, θ n,m,t is the tth time slot U n The elevation angle between and the mth PoI is expressed as: d n,m,t is the tth time slot U n The distance between the mth PoI is expressed as:

[0161]

[0162] make is the tth time slot U n The path loss of the LOS channel between and the mth PoI is expressed as:

[0163]

[0164] where α F is a constant that depends on the channel frequency and the speed of light, α LOS is the average additional line-of-sight path loss from the environment;

[0165] make is the tth time slot U n The path loss of the NLOS channel between and the m-th PoI is expressed as:

[0166]

[0167] where α NLOS is the average additional non-line-of-sight path loss from the environment;

[0168] Let h n,m,t is the tth time slot U n The average path loss between and the mth PoI is expressed as:

[0169]

[0170] Step 3: Model data collection rate.

[0171] Let γ n,m,t is the tth time slot U n The signal-to-noise ratio between the mth PoI and the mth PoI is expressed as:

[0172]

[0173] Where P m represents the transmission power of the mth PoI, and N0 represents the noise power spectral density at the UAV receiver;

[0174] Let R n,m,t is the tth time slot U n The data transmission rate between and the mth PoI is expressed as:

[0175] R n,m,t =Blog2(1+γ n,m,t ).

[0176] Step 5: Model the interest point detection probability and.

[0177] make For U n The corresponding received power when detecting the mth PoI in the tth time slot is expressed as:

[0178]

[0179] Where P s Indicates the transmission power used by the drone when detecting the target, G t represents the transmitting antenna gain of the UAV radar, G r represents the radar receiving antenna gain, σ m represents the radar cross-section of the mth PoI, λ represents the wavelength of the radar signal, L rt represents the loss coefficient;

[0180] make represents the tth time slot U n The detection probability corresponding to detecting the mth PoI is expressed as:

[0181]

[0182] where I0(·) represents the zero-order Bessel function, σ 2 Represents the noise power when the UAV perceives the target; V T Represents the signal amplitude threshold, let p fa Denotes the radar false alarm probability as a given constant, we can get:

[0183]

[0184] Let μ n,m,t ∈{0,1} is the PoI perception association variable, μ n,m,t =1 means U n The mth PoI is sensed in the tth time slot, and vice versa. n,m,t =0; by the end of the tth time slot, the mth PoI is at least n Probability of successful detection It is expressed as:

[0185]

[0186] The detection probability and p of all PoIs sum It is expressed as:

[0187]

[0188] Step 6: Model the system information age.

[0189] Let β n,m,t ∈{0,1} is the data collection variable of PoI, β n,m,t =1 means U n Collect the data of the mth PoI at the tth time slot, otherwise β n,m,t =0;

[0190] make The mth PoI transmits data to U in the tth time slot n The duration of is expressed as:

[0191]

[0192] Where D m,t is the remaining data amount to be transmitted to the UAV by the mth PoI at the end of the tth time slot, which is expressed as:

[0193]

[0194] D m represents the initial data volume of the mth PoI;

[0195] Let A m,t is the information age of the mth PoI in the tth time slot, which is expressed as:

[0196] A m,t =(ts m ) +

[0197] Among them, (x) + =max{0,x},s m is the time slot where the mth PoI starts to transmit data, which is expressed as:

[0198]

[0199] Let A represent the total information age of the system, which is expressed as:

[0200]

[0201] Step 7: Model the total energy consumption of the drone.

[0202] Let P f is the flight power of the UAV, which is expressed as:

[0203]

[0204] Among them, v tip represents the tip speed of the UAV rotor blade, v0 represents the average speed caused by the UAV rotor; constants c1, c2, c3 depend on the UAV power, rotor and air density;

[0205] Let p h is the hovering power of the drone per time slot, which is expressed as:

[0206] p h =c1+c2

[0207] make For UAV n Flight status variable indicator variable, Indicates U n In the tth time slot, it is in the flying state, otherwise it is in the hovering state.

[0208] Let E n,t For U n The energy consumption in the tth time slot is specifically expressed as:

[0209]

[0210] Let E n For UAV n The total energy consumption is specifically expressed as:

[0211]

[0212] Let E be the total energy consumption of the drone to collect data, which can be expressed as:

[0213]

[0214] Step 8: Model UAV mission scheduling and trajectory planning constraints.

[0215] 1) UAV-PoI association constraints include:

[0216]

[0217] 2) Target perception constraints include:

[0218]

[0219] where r e is the effective detection radius;

[0220]

[0221] where p this the detection probability threshold when starting data transmission;

[0222] 3) Data transmission restrictions include:

[0223]

[0224] 4) UAV trajectory restrictions include:

[0225] a) The flight position restrictions of the drone are:

[0226]

[0227] Among them, l th is the minimum safe distance between two drones;

[0228] b) The energy consumption limit of the drone is:

[0229]

[0230] in, Indicates drone U n The initial energy, Indicates U n The remaining energy threshold.

[0231] Step 9: Model the system state space, action space, and reward function.

[0232] (1) Let S t ∈S represents the system state space of the tth time slot, which is modeled as:

[0233]

[0234] in Denotes the UAV U in the tth time slot n status, The end time slot U n The remaining energy can be expressed as:

[0235]

[0236] represents the state of the mth PoI in the tth time slot, p m,t is the detection probability of the mth PoI by the end of the tth time slot, which is expressed as:

[0237]

[0238] (2) Let A t ∈A represents the action space of the tth time slot, then: A t ={a 1,t,…,a n,t ,…,a N,t}, where a n,t Denotes the UAV U in the tth time slot n The action space,

[0239] (3) Let R t ∈R represents the reward function of the tth time slot, then R t = {r 1,t ,…,r n,t ,…,r N,t}, r n,t Denotes the UAV U in the tth time slot n In status n,t When selecting action a n,t The rewards obtained are specifically modeled as follows:

[0240]

[0241] Among them, ω1, ω2, ω3, ω4 represent weight factors, Denotes the UAV U in the tth time slot n Data collection rewards,

[0242]

[0243] Denotes the UAV U in the tth time slot n Perceived Rewards:

[0244]

[0245] Denotes the UAV U in the tth time slot n The associated data collection task information age penalty items are:

[0246]

[0247] C1 and C2 are positive numbers, representing the penalty for the drone returning to the starting point without collecting all the data, and the penalty for not returning to the starting point when the remaining battery power of the drone is below the threshold.

[0248] Step 10: Model and train the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) with Long Short-Term Memory (LSTM) and Attention Mechanism.

[0249] (1) Constructing a MADDPG network based on LSTM and attention mechanism: The constructed network includes a real actor network with parameter ω, a real critic network with parameter ω′, a target actor network with parameter θ, and a target critic network with parameter θ′; the fully connected layer FCN in the four networks consists of a cascaded LSTM layer and a soft attention layer;

[0250] In the cascaded LSTM layer, let f t represents the forget gate of the tth time slot, c t represents the cell state of the tth time slot, h t represents the hidden state of the tth time slot, which is specifically expressed as:

[0251] f t =σ l (W f ·[x t +h t-1 ]+b f )

[0252] c t =tanh(W c ·[x t +h t-1 ]+b c )*i t +f t *c t-1 h t =o t *tanh(c t )

[0253] Among them, i t represents the input gate of the tth time slot; o t represents the output gate of the tth time slot; W f , W c , b f , b c represents network parameters; h t-1 The hidden state of the t-1th time slot; c t-1 represents the cell state of the t-1th time slot; * represents the Hadamard product of the matrix; σ l Represents the sigmoid activation function;

[0254] In the soft attention layer, let Indicates the tth time slot and time slot The attention weight between is expressed as:

[0255]

[0256] in, represents the tth time slot ht and c t The state summary vector u t The transpose of

[0257] Let h t ′ represents the updated hidden state of the tth time slot, which is expressed as:

[0258]

[0259] (2) Initialize network parameters: The current Actor network interacts with the environment, inputs state S to the real Actor network to obtain action A, applies action A to the environment, switches to state S′ at the next moment, and obtains reward R; stores the four-tuple (s, a, r, s′) in the experience pool;

[0260] (3) Sampling and parameter training: Take samples from the experience pool for training and transform the state S of the tth time slot t and action A t Input into the real Critic network to calculate the current Q(S t ,A t ; θ) value; the state S at the next moment t+1 Input to the target Actor network to get action A t+1 ; Change state S t+1 and action A t+1 Input into the target Critic network to calculate the target value; order Represents the loss function between two Q values. By minimizing the loss function, the parameter ω′ of the real critic network is updated;

[0261] (4) Update the actual Actor network parameters ω based on the policy gradient method:

[0262]

[0263] Where J is the agent's objective function, represents the gradient of f(x) with respect to x; μ(S t ,A t ; ω) represents the strategy of mapping the real Actor network from state to action; α is the learning rate of the gradient descent algorithm, 0≤α≤1;

[0264] (5) Update the network parameters based on the soft update method: Use the parameters ω of the actual actor network and ω′ of the actual critic network to update the parameters θ of the target actor network and θ′ of the target critic network, which can be expressed as:

[0265] ω′←ρω+(1-ρ)ω′

[0266] θ′←ρθ+(1-ρ)θ′

[0267] Where ρ represents the learning rate, ρ∈(0,1) and ρ<<1;

[0268] (6) Repeat the training process: first determine whether the termination condition has been reached. If not, return to step (2) and use the real Actor network to sample, and repeat this cycle until the termination condition is met. After the training is completed, the result obtained by the algorithm is the real Actor network. In the actual decision-making, when a state is input, the Actor network will output an optimal action. The other three networks are the optimal auxiliary networks obtained through training.

[0269] Step 11: Using the trained MADDPG network combining LSTM and attention mechanism, the flight strategy, perception scheduling and data collection strategy of the drone are determined with the goal of optimizing the reward function while satisfying the drone’s perception and data collection constraints.

[0270] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A drone-assisted environmental state perception and data collection method, characterized in that: The method comprises the following steps: S1. Build UAV-assisted environmental state perception and data collection scenarios, and model the system model; S2, establish the UAV channel model and model the data collection rate in the UAV-assisted environmental state perception and data collection scenario; S3, modeling interest point detection probability and; S4, modeling system information age; S5, total energy consumption of modeled UAV; S6, modeling drone perception and data collection constraints; S7. Modeling system state space, action space and reward function S8. Model and train a multi-agent deep deterministic policy gradient model combining long short-term memory and attention mechanism; S9. Determine the flight trajectory, perception scheduling and data collection strategy of the drone based on the trained model.

2. The method for environmental state perception and data collection assisted by a drone according to claim 1, characterized in that: In step S1, the drone-assisted environmental state perception and data collection scenario includes multiple drones, a ground base station and multiple ground points of interest (PoIs); the PoIs collect environmental state information, and the drone detects the state of the PoIs. If the detection is successful, the data of the PoIs is collected; Let U n represents the nth drone, 1≤n≤N; divide the system time into T time slots of equal length, and the time slot length is τ; let represents the tth time slot U n The flight angle, make represents the tth time slot U n The position of H is U n The flight altitude of the drone is expressed as follows: Where v is the flight speed of the drone; make represents the position of the mth PoI, 1≤m≤M, M is the total number of PoIs; PoI transmits data to the UAV through frequency division multiple access OFDMA technology, and the bandwidth of each subchannel is B.

3. The method for environmental state perception and data collection assisted by a drone according to claim 2, characterized in that: In step S2, the UAV channel model is defined as: The channel between the UAV and the PoI is modeled as a probabilistic line-of-sight LOS channel. represents the tth time slot U n The probability that the channel between and the mth PoI is a LOS channel is expressed as: where a and a′ are constants related to the environment, θ n,m,t is the tth time slot U n The elevation angle between and the mth PoI is expressed as: d n,m,t is the tth time slot U n The distance between the mth PoI is expressed as: make is the tth time slot U n The path loss of the LOS channel between and the mth PoI is expressed as: where α F is a constant, α LOS is the random fading coefficient of the LOS link; make is the tth time slot U n The path loss of the NLOS channel between and the m-th PoI is expressed as: where α NLOS is the random fading coefficient of the NLOS link; Let h n,m,t is the tth time slot U n The average path loss between and the mth PoI is expressed as:

4. The method for environmental state perception and data collection assisted by a drone according to claim 3, characterized in that: In step S2, the data collection rate is modeled as follows: Let γ n,m,t is the tth time slot U n The signal-to-noise ratio between the mth PoI and the mth PoI is expressed as: Where P m represents the transmission power of the mth PoI, and N0 represents the noise power spectral density at the UAV receiver; Let R n,m,t is the tth time slot U n The data transmission rate between and the mth PoI is expressed as: R n,m,t =Blog2(1+γ n,m,t )。 5. The method for environmental state perception and data collection assisted by a drone according to claim 4, characterized in that: In step S3, the interest point detection probability and are modeled as follows: make For U n The corresponding received power when detecting the mth PoI in the tth time slot is expressed as: Where P s Indicates the transmission power used by the drone when detecting the target, G t represents the transmitting antenna gain of the UAV radar, G r represents the radar receiving antenna gain, σ m represents the radar cross-section of the mth PoI, λ represents the wavelength of the radar signal, L rt represents the loss coefficient; make represents the tth time slot U n The detection probability corresponding to detecting the mth PoI is expressed as: where I0(·) represents the zero-order Bessel function, σ 2 Represents the noise power when the UAV perceives the target; V T Represents the signal amplitude threshold, let p fa Denotes the radar false alarm probability as a given constant, we can get: Let μ n,m,t ∈{0,1} is the PoI perception association variable, μ n,m,t =1 means U n The mth PoI is sensed in the tth time slot, and μ n,m,t =0; by the end of the tth time slot, the mth PoI is at least n Probability of successful detection It is expressed as: The detection probability and p of all PoIs sum It is expressed as:

6. The method for environmental state perception and data collection assisted by a drone according to claim 5, characterized in that: In step S4, the system information age is modeled, specifically: Let β n,m,t ∈{0,1} is the data collection variable of PoI, β n,m,t =1 means U n Collect the data of the mth PoI at the tth time slot, otherwise β n,m,t =0; make The mth PoI transmits data to U in the tth time slot n The duration of is expressed as: Where D m,t is the remaining data amount to be transmitted to the UAV by the mth PoI at the end of the tth time slot, which is expressed as: D m represents the initial data volume of the mth PoI; Let A m,t is the information age of the mth PoI in the tth time slot, which is expressed as: A m,t =(t-s m ) + Among them, (x) + =max{0,x},s m is the time slot where the mth PoI starts to transmit data, which is expressed as: Let A represent the total information age of the system, which is expressed as:

7. The method for environmental state perception and data collection assisted by a drone according to claim 6, characterized in that: In step S5, the total energy consumption of the UAV is modeled: Let P f is the flight power of the UAV, which is expressed as: Among them, v tip represents the tip speed of the UAV rotor blade, v0 represents the average speed caused by the UAV rotor; constants c1, c2, c3 depend on the UAV power, rotor and air density; Let p h is the hovering power of the drone per time slot, which is expressed as: p h =c1+c2 make For UAV n Flight status variable indicator variable, Indicates U n It is in the flying state at the tth time slot, otherwise it is in the hovering state; Let E n,t For U n The energy consumption in the tth time slot is specifically expressed as: Let E n For UAV n The total energy consumption is specifically expressed as: Let E be the total energy consumption of the drone to collect data, which can be expressed as:

8. The method for environmental state perception and data collection assisted by a drone according to claim 7, characterized in that: In step S6, the constraints of UAV perception and data collection include UAV-PoI association constraints, perception constraints, data transmission constraints, and UAV flight constraints, where: 1) UAV-PoI association constraints include: 2) Target perception constraints include: where r e is the effective detection radius; where p th is the detection probability threshold when starting data transmission; 3) Data transmission restrictions include: 4) UAV trajectory restrictions include: a) The flight position restrictions of the drone are: Among them, l th is the minimum safe distance between two drones; b) The energy consumption limit of the drone is: in, Indicates drone U n The initial energy, Indicates U n The remaining energy threshold.

9. The method for environmental state perception and data collection assisted by a drone according to claim 8, characterized in that: In step S7, the system state space, action space and reward function are modeled, including: (1) Let S t ∈S represents the system state space of the tth time slot, which is modeled as: in Denotes the UAV U in the tth time slot n status, The end time slot U n The remaining energy can be expressed as: represents the state of the mth PoI in the tth time slot, p m,t is the detection probability of the mth PoI by the end of the tth time slot, which is expressed as: (2) Let A t ∈A represents the action space of the tth time slot, then: A t ={a 1,t ,…,a n,t ,…,a N,t }, where an,t represents the UAV U in the tth time slot n The action space, (3) Let R t ∈R represents the reward function of the tth time slot, then R t = {r 1,t ,…,r n,t ,…,r N,t }, r n,t Denotes the UAV U in the tth time slot n In status n,t When selecting action a n,t The rewards obtained are specifically modeled as follows: Among them, ω1, ω2, ω3, ω4 represent weight factors, Denotes the UAV U in the tth time slot n Data collection rewards, Denotes the UAV U in the tth time slot n Perceived Rewards: Denotes the UAV U in the tth time slot n The associated data collection task information age penalty items are: C1 and C2 are positive numbers, representing the penalty for the drone returning to the starting point without collecting all the data, and the penalty for not returning to the starting point when the remaining battery power of the drone is below the threshold.

10. The method for environmental state perception and data collection assisted by a drone according to claim 9, characterized in that: In step S8, a multi-agent deep deterministic policy gradient (MADDPG) combining long short-term memory (LSTM) and attention mechanism is modeled and trained, specifically including: (1) Constructing a MADDPG network based on LSTM and attention mechanism: The constructed network includes a real actor network with parameter ω, a real critic network with parameter ω′, a target actor network with parameter θ, and a target critic network with parameter θ′; the fully connected layer FCN in the four networks consists of a cascaded LSTM layer and a soft attention layer; In the cascaded LSTM layer, let f t represents the forget gate of the tth time slot, c t represents the cell state of the tth time slot, h t represents the hidden state of the tth time slot, which is specifically expressed as: f t =σ l (W f ·[x t +h t-1 ]+b f ) c t =tanh(W c ·[x t +h t-1 ]+b c )*i t +f t *c t-1 h t =o t *tanh(c t ) Among them, i t represents the input gate of the tth time slot; o t represents the output gate of the tth time slot; W f , W c , b f , b c represents network parameters; h t-1 The hidden state of the t-1th time slot; c t-1 represents the cell state of the t-1th time slot; * represents the Hadamard product of the matrix; σ l Represents the sigmoid activation function; In the soft attention layer, let Indicates the tth time slot and time slot The attention weight between is expressed as: in, represents the tth time slot h t and c t The state summary vector u t The transpose of Let h t ′ represents the updated hidden state of the tth time slot, which is expressed as: (2) Initialize network parameters: The current Actor network interacts with the environment, inputs state S to the real Actor network to obtain action A, applies action A to the environment, switches to state S′ at the next moment, and obtains reward R; stores the four-tuple (s, a, r, s′) in the experience pool; (3) Sampling and parameter training: Take samples from the experience pool for training and transform the state S of the tth time slot t and action A t Input into the real Critic network to calculate the current Q(S t ,A t ; θ) value; the state S at the next moment t+1 Input to the target Actor network to get action A t+1 ; Change state S t+1 and action A t+1 Input into the target Critic network to calculate the target value; order Represents the loss function between two Q values. By minimizing the loss function, the parameter ω′ of the real critic network is updated; (4) Update the actual Actor network parameters ω based on the policy gradient method: Where J is the agent's objective function, represents the gradient of f(x) with respect to x; μ(S t ,A t ; ω) represents the strategy of mapping the real Actor network from state to action; α is the learning rate of the gradient descent algorithm, 0≤α≤1; (5) Update the network parameters based on the soft update method: Use the parameters ω of the actual actor network and ω′ of the actual critic network to update the parameters θ of the target actor network and θ′ of the target critic network, which can be expressed as: ω′←ρω+(1-ρ)ω′ θ′←ρθ+(1-ρ)θ′ Where ρ represents the learning rate, ρ∈(0,1) and ρ<<1; (6) Repeat the training process: first determine whether the termination condition has been reached. If not, return to step (2) and use the real Actor network to sample, and repeat this cycle until the termination condition is met. After the training is completed, the result obtained by the algorithm is the real Actor network. In the actual decision-making, when a state is input, the Actor network will output an optimal action. The other three networks are the optimal auxiliary networks obtained through training.

11. The method for environmental state perception and data collection assisted by a drone according to claim 10, characterized in that: In step S9, the trained MADDPG network combining LSTM and attention mechanism is used to determine the flight strategy, perception scheduling and data collection strategy of the drone with the goal of optimizing the reward function while satisfying the drone perception and data collection constraints.

Images