A machine learning training method for UAV communication and perception integrated network

Abstract

The present invention discloses a machine learning training method for an integrated UAV communication and perception network, relating to the field of UAV control technology. The UAV communication and perception integrated network includes an action neural network and an evaluation neural network, and is trained using a continuous control algorithm. The action neural network takes the environment state space as input and the action as output. The evaluation neural network takes the environment state space and the action as input and outputs the reward obtained by the UAV during single-step training. The reward consists of the UAV's terminal reward, process perception reward, safe flight reward, process energy reward, flight guidance reward, and penalty for flying out of the valid motion space. When the UAV flies out of the valid motion space or the given UAV onboard energy is exhausted, the training of the current round of the UAV communication and perception integrated network is terminated. The present invention can guide deep reinforcement learning training and achieve joint planning of UAV communication, perception, and control.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle (UAV) control technology, and in particular to a machine learning training method for an UAV communication and perception integrated network. Background Art

[0002] Integrated communication and perception technology is one of the six scenarios used in future 6G wireless networks. It not only provides high-throughput and low-latency communication services but also generates highly accurate perception capabilities. As a flexible and cost-effective aerial platform, drones (UAVs) have been widely used in military and civilian fields, such as reconnaissance, disaster relief, traffic monitoring, agriculture, forestry, animal husbandry, and logistics. In recent years, the combination of UAVs and integrated communication and perception technology has attracted increasing research attention.

[0003] To fully exploit the potential of drone-synaesthesia integration, a wealth of research is currently underway on the optimal design of drone networks, including drone trajectory planning and synaesthesia resource allocation. However, most research relies primarily on numerical optimization methods to achieve suboptimal solutions. This approach presents several challenges, including reliance on precise mathematical models, high computational complexity, a strong dependency on specific hardware, and an inability to adapt to dynamic environments. Consequently, translating these theoretical advances into practical applications of drone-synaesthesia integrated systems presents significant challenges. Summary of the Invention

[0004] The present invention provides a machine learning training method for an integrated UAV communication and perception network, which can solve the above-mentioned technical problems.

[0005] The technical solution adopted by the present invention is as follows:

[0006] The present invention provides a machine learning training method for an integrated UAV communication and perception network. The integrated UAV communication and perception network includes an action neural network and an evaluation neural network, and is trained using a continuous control algorithm.

[0007] The input of the action neural network is the environment state space s(t), and the output is the action a(t);

[0008] Environmental state space

[0009]

[0010] Among them, x UB (t),y UB (t),z UB (t) are the relative positions of the UAV and the base station in the x, y, and z directions in the fixed ground coordinates, v x (t),v y (t),v z(t) are the current velocity vectors of the UAV in the x, y, and z directions, respectively, and ω x (t),ω y (t),ω z (t) are the current angular velocities of the drone in the x, y, and z directions, respectively. q0(t), q1(t), q2(t), and q3(t) are quaternions used to describe the drone’s posture. E r (t) is the current remaining energy of the UAV, Q r (t) is the current perception information throughput of the UAV,

[0011] x UF (t),y UF (t),z UF (t) are the relative positions of the UAV and the endpoint in the x, y, and z directions based on the fixed ground coordinates;

[0012] x UT (t),y UT (t),z UT (t) are the relative positions of the UAV and the moving target in the x, y, and z directions in the ground fixed coordinates;

[0013] V T is the moving speed of the moving target;

[0014] Action a(t) = {u1(t), u2(t), u3(t), u4(t), ρ(t)}, where {u1(t), u2(t), u3(t), u4(t)} are control variables and ρ(t) is the proportion of communication perception time.

[0015] The input of the evaluation neural network is the environment state space s(t) and the action a(t), and the output is the reward r obtained by the drone in a single step of training, r = r d +r s +r h +r e +r g +r 3d , where r d is the terminal reward of the drone, r s is the process perception reward of the UAV, r h Reward for safe flight of drones, r e is the process energy reward of the drone, r g Reward for drone flight guidance, r 3d Penalties for drones flying outside of their effective movement space;

[0016] When the UAV flies out of the effective motion space or the given UAV's onboard energy is exhausted, the current round of training of the UAV communication and perception integrated network is terminated.

[0017] In a preferred embodiment of the present invention, the continuous control algorithm is Deep Deterministic Policy Gradient (DDPG), or Double Delayed Deep Deterministic Policy Gradient (TD3), or Soft Policy Gradient (SAC).

[0018] In a preferred embodiment of the present invention, the action neural network includes 1 input layer, 6 hidden layers and 1 output layer; the input layer of the action neural network includes 23 neurons, which are used to input the environment state space s(t), and the ReLu function is used as the activation function; the hidden layers of the action neural network are connected in a fully connected manner, and the number of neurons is 128, 256, 256, 256, 256 and 128 respectively, and they are all fully connected layers, and the ReLu function is used as the activation function; the output layer of the action neural network includes 5 neurons, and the output results correspond to the 4 control quantities and communication perception time proportion of the drone, and the tanh function is used as the activation function.

[0019] In a preferred embodiment of the present invention, the network structure of the evaluation neural network includes 1 input layer, 6 hidden layers, and 1 output layer, and the layers are fully connected; the input layer of the evaluation neural network contains 27 neurons, which are used to input the environment state space s(t) and action a(t), and the ReLu function is used as the activation function; the hidden layers of the evaluation neural network are connected in a fully connected manner, and the number of neurons is 128, 256, 256, 256, 256 and 128 respectively, and they are all fully connected layers, and the ReLu function is used as the activation function; the output layer of the evaluation neural network contains only 1 neuron, which is used to output the reward r obtained by the drone in single-step training.

[0020] In a preferred embodiment of the present invention, the terminal reward of the drone

[0021]

[0022] Among them, d d =||qx F ||, q=(x(t),y(t),z(t)) is the current position coordinate of the drone, x F =(x F ,y F ,z F ) is the end position; ξ d is the critical distance; c d is the maximum terminal distance reward; Q r is the total amount of perceptual mutual information;

[0023] ε d , ε d1 and ε d2 are all positive real constant parameters, 1>εd >0, 1>ε d1 >0, 1>ε d2 >0;

[0024]

[0025] When d d >ξ d When the UAV is far away from the destination, it needs to be penalized for the distance and rewarded with a discount on the total amount of perceived mutual information. When the UAV reaches the destination, it can obtain all terminal rewards.

[0026] Process Perception Rewards for Drones

[0027]

[0028] Among them, ε s1 and ε s2 is a positive real constant parameter and satisfies ε s1 +ε s2 =1, R c is the communication transmission rate, R r is the perceived information rate, R v is the image data generation rate, k r is a positive real constant weight parameter, E tot The total energy available for drone flight.

[0029] Drone Safe Flight Rewards

[0030] r h =c h +k h min(zh min ,0)+k h min(h max -z,0)

[0031] Among them, c h For safe flight rewards, k h is the height penalty weight, h min and h max They are the minimum and maximum flight altitudes respectively.

[0032] Drone Process Energy Bonus

[0033] r e =k e (E r (t)-E r (t-dt))

[0034] Among them, k e is the energy penalty weight, and dt is the time step.

[0035] Drone flight guidance rewards

[0036]

[0037] Among them, ξ e is the critical energy value, d T,t-1 The distance between the drone and the perceived target in the last state sampling, d T,t The distance between the current state sampling drone and the sensing target, d d,t-1 The distance between the last state sampling drone and the end point of the position, d d,t The distance between the current state sampling drone and the end point of the position.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] By designing a specific neural network and reward function, the present invention can guide neural network training to complete communication perception tasks with energy constraints and safe flight constraints. The reward for single-step training is the sum of the drone's terminal reward, process perception reward, safe flight reward, process energy reward, flight guidance reward, and the penalty for the drone flying out of the valid motion space.

[0040] The accumulated data of the drone's interaction with the environment is used to train the neural network and fit the potential causal relationship, avoiding the reliance on precise mathematical models;

[0041] The trained action neural network can directly output the drone's control instructions and the distribution of communication and perception time, avoiding highly complex optimization calculations and adapting to dynamic environmental changes.

[0042] Neural network calculations only require four simple arithmetic operations, reducing dependence on specific hardware devices.

[0043] In order to make the above-mentioned objects, features and advantages of the present invention more clearly understood, embodiments of the present invention are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 It is a flow chart of the machine learning training method for the integrated communication and perception network of UAVs;

[0046] Figure 2 This is the architecture diagram of the action neural network;

[0047] Figure 3 This is the architecture diagram of the evaluation neural network. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0049] The present invention provides a machine learning training method for an integrated UAV communication and perception network, which is as follows:

[0050] like Figure 1 As shown, the elements in the environment include drones, base stations, and moving targets.

[0051] For drones, the state variables that affect control decisions are:

[0052] Current position coordinates (x(t), y(t), z(t)), where x(t), y(t), and z(t) are the three-dimensional coordinates of the drone at time t in the fixed ground coordinates.

[0053] Current velocity vector (v x (t),v y (t),v z (t)), vx(t), v y (t) and vz(t) are the flight speeds of the UAV in the three-dimensional coordinate axis direction at the fixed ground coordinate at time t;

[0054] Current body rotation angular velocity (ω x (t),ω y (t),ω z (t)),ω x (t),ω y (t) and ω z (t) is the rotation speed of the UAV in the three-dimensional coordinate axis direction in the body coordinate at time t;

[0055] Quaternion (q0(t),q1(t),q2(t),q3(t)) used to describe the body posture, where q0(t) is the value of the real part,

[0056] q1(t),q2(t),q3(t) are the values ​​of the three imaginary parts respectively;

[0057] Current remaining energy E r (t), perceived information throughput Q r (t);

[0058] There are 15 status variables in total.

[0059] For the base station, the state quantity that affects the control decision is the base station position coordinate (x B ,y B ,z B ), x B ,y B and z B are the three-dimensional coordinates of the base station under the fixed coordinates on the ground.

[0060] For a moving target, the state quantity that affects the control decision is:

[0061] The current position of the moving target (x T (t),y T (t),z T (t)), x T (t), y T (t) and z T (t) are the three-dimensional coordinates of the target at the fixed ground coordinate at time t; and the moving target speed (V T x,V T y), v T x(t) and v Ty (t) are the moving speeds of the target along the ground plane coordinate axis at the fixed ground coordinate at time t.

[0062] For the flight process, it is necessary to clarify the end point position coordinates (x F ,y F ,z F ), x F ,y F and z F are the three-dimensional coordinates of the destination in fixed ground coordinates.

[0063] Therefore, there are a total of 26 state quantities in the environment that affect the drone's control decision. Through analysis, it is found that the training purpose of the present invention is only related to the relative position of the drone and the base station, and the relative position of the drone and the moving target. If the relative position is used to represent the environmental state, three state quantities can be reduced, which is beneficial for deep reinforcement learning. Therefore, the present invention adopts

[0064] Relative position of drone and base station

[0065] (x UB (t),y UB (t),z UB (t))=(x(t)-x B ,y(t)-y B ,z(t)-z B )

[0066] The relative position of the drone and the moving target

[0067] (x UT (t),y UT (t),z UT (t))=(x(t)-x T (t),y(t)-y T (t),z(t)-z T (t))

[0068] The relative position of the drone and the end point

[0069] (x UF (t),y UF (t),z UF (t))=(x(t)-x F ,y(t)-y F ,z(t)-z F )

[0070] To characterize,

[0071] The environment state space is

[0072]

[0073] All observations of environmental states need to be normalized to ensure that their values ​​are in the range [-1,1].

[0074] In the present invention, the drone uses four motors to drive the propellers to generate pulling force to change the position and posture of the drone. The drone's action a(t) = {u1(t), u2(t), u3(t), u4(t), ρ(t)}, among which {u1(t), u2(t), u3(t), u4(t)} are control quantities, and ρ(t) is the proportion of communication perception time.

[0075] The UAV communication and perception integrated network includes an action neural network and an evaluation neural network, and is trained using a continuous control algorithm such as Deep Deterministic Policy Gradient (DDPG), or Double Delayed Deep Deterministic Policy Gradient (TD3), or Soft Policy Gradient (SAC).

[0076] like Figure 2As shown in the figure, the input of the action neural network is the environment state space s(t), and the output is the action a(t). The action neural network consists of 1 input layer, 6 hidden layers, and 1 output layer. The input layer of the action neural network contains 23 neurons for inputting the environment state space s(t), and uses the ReLu function as the activation function. The number of neurons in each hidden layer of the action neural network is 128, 256, 256, 256, 256, and 128 respectively. All are fully connected layers and use the ReLu function as the activation function. The output layer of the action neural network contains 5 neurons, and the output results correspond to the four control variables of the drone and the proportion of communication perception time. The tanh function is used as the activation function.

[0077] like Figure 3 As shown in the figure, the input of the evaluation neural network is the environment state space s(t) and the action a(t), and the output is the reward r obtained by the drone in a single-step training. The network structure of the evaluation neural network includes 1 input layer, 6 hidden layers, and 1 output layer, and each layer is fully connected. The input layer of the evaluation neural network contains 27 neurons for inputting the environment state space s(t) and the action a(t), and uses the ReLu function as the activation function. The number of neurons in each hidden layer of the evaluation neural network is 128, 256, 256, 256, 256, and 128 respectively, and all are fully connected layers, and all use the ReLu function as the activation function. The output layer of the evaluation neural network contains only 1 neuron, which is used to output the reward r obtained by the drone in a single-step training.

[0078] The reward obtained by the drone in a single-step training

[0079] r=r d +r s +r h +r e +r g +r 3d

[0080] Among them, r d is the terminal reward of the drone, r s is the process perception reward of the UAV, r h Reward for safe flight of drones, r e is the process energy reward of the drone, r g Reward for drone flight guidance, r 3d It is a penalty for the UAV to fly out of the effective movement space; when the UAV flies out of the effective movement space or the given UAV airborne energy is exhausted, the training of the current round of the UAV communication and perception integrated network is terminated.

[0081] The following is an explanation of the reward r obtained by the drone in single-step training:

[0082] (1) UAV terminal rewards

[0083] The training goal of the present invention is that the drone flies to the designated destination before the energy is exhausted, and the more mutual information about the target being measured is perceived during the flight, the better. Therefore, the design of the terminal reward is related to the distance between the destinations, the remaining energy, and the total amount of mutual information perception. The drone can perceive more targets by making full use of the energy. Therefore, the algorithm greedily assumes that the drone will return to the destination just when the given energy is exhausted. Based on the above considerations, when the drone's given energy is exhausted, the drone's terminal reward is designed to be

[0084]

[0085] Among them, d d =||qx F ||, q=(x(t),y(t),z(t)) is the current position coordinate of the drone, x F =(x F ,y F ,z F ) is the end position; ξ d is the critical distance; c d is the maximum terminal distance reward; Q r is the total amount of perceptual mutual information;

[0086] ε d , ε d1 and ε d2 They are all positive real constant parameters and need to be pre-set according to actual conditions. 1>ε d >0, 1>ε d1 >0, 1>ε d2 >0;

[0087] When d d >ξ d When the UAV is far away from the destination, it needs to be penalized for the distance and the total amount of mutual information is discounted. When the UAV reaches the destination (d d =0), the drone can obtain all terminal rewards, i.e. r d =c d +Q r , Q r The bigger the r d To ensure the progressiveness of the reward,

[0088]

[0089] Because Q r The value is uncertain, and constant parameters need to be set based on parameter estimates.

[0090] (2) UAV process perception reward

[0091] In the process reward, the communication transmission capacity is required to be greater than the sum of the radio perception and video perception rates, that is, the communication can transmit the perception content in real time. However, as the onboard energy decreases, the reward obtained by process perception should be reduced accordingly to encourage the drone to fly to the destination. In view of the above considerations, the process perception reward of the drone is designed to be

[0092]

[0093] Among them, ε s1 and ε s2 is a positive real constant parameter and satisfies ε s1 +ε s2 =1, R c is the communication transmission rate, R r is the perceived information rate, R v is the image data generation rate, k r is a positive real constant weight parameter, E tot The total energy available for drone flight.

[0094] (3) Drone safe flight rewards

[0095] Drones that violate safe flight altitudes will be penalized, and those that maintain safe flight will be rewarded to encourage continued flight.

[0096] r h =c h +k h min(z(t)-h min ,0)+k h min(h max -z(t),0)

[0097] Among them, c h For safe flight rewards, k h is the height penalty weight, h min and h max They are the minimum and maximum flight altitudes respectively.

[0098] (4) Drone process energy reward

[0099] The energy consumption of a drone in a single-step flight is too large and should be punished to discourage high-power maneuvers and increase the drone's flight time. The process energy reward of the drone is designed as follows:

[0100] r e =k e (E r (t)-E r (t-dt))

[0101] Among them, k e is the energy penalty weight, and dt is the time step.

[0102] (5) UAV flight guidance rewards

[0103] According to intuitive observation, when the drone has sufficient energy, it should fly towards the target, shorten the sensing distance, and enhance the sensing ability; when the drone has less remaining energy, it should fly towards the end point to ensure that it can reach the end point before the energy is exhausted. Based on the above considerations, the drone's flight guidance reward is designed as follows:

[0104]

[0105] Among them, ξ e is the critical energy value, d T,t-1 The distance between the drone and the perceived target in the last state sampling, d T,t The distance between the current state sampling drone and the sensing target, d d,t-1 The distance between the last state sampling drone and the end point of the position, d d,t The distance between the drone and the end point of the current state sampling. Through the above settings, when the remaining airborne energy of the drone is greater than the critical energy, the drone will be rewarded if it approaches the perception target, and will not be punished if it is far away from the perception target. This design is conducive to avoiding ξ e When the setting is unreasonable, the drone needs to fly to the destination in advance without being punished. The same applies to the guidance reward at the destination. e When the settings are unreasonable, the drone can use more energy to perceive the target without being penalized.

[0106] (6) Drone movement space and round end settings

[0107] Setting an effective UAV motion space can prevent the UAV from performing ineffective exploration, such as the UAV continuously decreasing its altitude or continuously flying in a direction that violates the mission objective. The UAV motion space should be larger than the space required for normal flight in the mission scenario. Set the UAV's three-dimensional motion space as Ω 3d ={[x min ,x max ];[y min ,y max ];[z min ,z max ]}, where x min ,y min ,z min is the lower boundary value of the space, x max ,y max ,z max is the upper boundary value of the space.

[0108] If the drone flies out of the movement space, the training round will be terminated and a penalty will be given.

[0109] r 3d =-c 3d

[0110] where c 3d is the penalty value for violating the boundary.

[0111] When the UAV flies out of the motion space or the given UAV's onboard energy is exhausted, the training round ends and the end flag is set to done = 1, otherwise done = 0.

[0112] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A machine learning training method for an integrated UAV communication and perception network, characterized in that: The UAV communication and perception integrated network includes an action neural network and an evaluation neural network, and is trained using a continuous control algorithm. The input of the action neural network is the environment state space s(t), and the output is the action a(t); Environmental state space Among them, x UB (t),y UB (t),z UB (t) are the relative positions of the UAV and the base station in the x, y, and z directions, respectively, and v x (t),v y (t),v z (t) are the current velocity vectors of the UAV in the x, y, and z directions, respectively, and ω x (t),ω y (t),ω z (t) are the current angular velocities of the drone in the x, y, and z directions, respectively. q0(t), q1(t), q2(t), and q3(t) are quaternions used to describe the drone’s posture. E r (t) is the current remaining energy of the UAV, Q r (t) is the current perception information throughput of the UAV, x UF (t),y UF (t),z UF (t) are the relative positions of the UAV and the endpoint in the x, y, and z directions, respectively; x UT (t),y UT (t),z UT (t) are the relative positions of the UAV and the moving target in the x, y, and z directions respectively; V T is the moving speed of the moving target; Action a(t) = {u1(t), u2(t), u3(t), u4(t), ρ(t)}, where {u1(t), u2(t), u3(t), u4(t)} are control variables and ρ(t) is the proportion of communication perception time. The input of the evaluation neural network is the environment state space s(t) and the action a(t), and the output is the reward r obtained by the drone in a single-step training, r = r d +r s +r h +r e +r g +r 3d , where r d is the terminal reward of the drone, r s is the process perception reward of the UAV, r h Reward for safe flight of drones, r e is the process energy reward of the drone, r g Reward for drone flight guidance, r 3d Penalties for drones flying outside of their effective movement space; When the UAV flies out of the effective motion space or the given UAV's onboard energy is exhausted, the current round of training of the UAV communication and perception integrated network is terminated.

2. The machine learning training method for the UAV communication and perception integrated network according to claim 1 is characterized in that: The continuous control algorithm is a deep deterministic policy gradient algorithm, a double-delayed deep deterministic policy gradient algorithm, or a soft policy gradient algorithm.

3. The machine learning training method for the UAV communication and perception integrated network according to claim 1 is characterized in that: The action neural network consists of 1 input layer, 6 hidden layers and 1 output layer; the input layer of the action neural network contains 23 neurons, which are used to input the environment state space s(t), and the ReLu function is used as the activation function; the number of neurons in each hidden layer of the action neural network is 128, 256, 256, 256, 256 and 128 respectively, and all are fully connected layers, and the ReLu function is used as the activation function; the output layer of the action neural network contains 4 neurons, and the output results correspond to the 4 control variables of the drone, and the tanh function is used as the activation function.

4. The machine learning training method for the UAV communication and perception integrated network according to claim 1 is characterized in that: The network structure of the evaluation neural network includes 1 input layer, 6 hidden layers, and 1 output layer, and each layer is fully connected; the input layer of the evaluation neural network contains 27 neurons, which are used to input the environment state space s(t) and action a(t), and the ReLu function is used as the activation function; the number of neurons in each hidden layer of the evaluation neural network is 128, 256, 256, 256, 256 and 128 respectively, and all are fully connected layers, and all use the ReLu function as the activation function; the output layer of the evaluation neural network contains only 1 neuron, which is used to output the reward r obtained by the drone in single-step training.

5. The machine learning training method for the UAV communication and perception integrated network according to claim 1 is characterized in that: Terminal rewards for drones Among them, d d =||qx F ||, q=(x(t),y(t),z(t)) is the current position coordinate of the drone, x F =(x F ,y F ,z F ) is the end position; ξ d is the critical distance; c d is the maximum terminal distance reward; Q r is the total amount of perceptual mutual information; ε d , ε d1 and ε d2 All are normal parameters, 1>ε d >0, 1>ε d1 >0, 1>ε d2 >0; When d d >ξ d When the UAV is far away from the destination, it needs to be penalized for the distance and rewarded with a discount on the total amount of perceived mutual information. When the UAV reaches the destination, it can obtain all terminal rewards.

6. The machine learning training method for the UAV communication and perception integrated network according to claim 5, characterized in that: Process Perception Rewards for Drones Among them, ε s1 and ε s2 is a positive real constant parameter and satisfies ε s1 +ε s2 =1, R c is the communication transmission rate, R r is the perceived information rate, R v is the image data generation rate, k r is a positive real constant weight parameter, E tot The total energy available for drone flight.

7. The machine learning training method for the UAV communication and perception integrated network according to claim 6, characterized in that: Drone Safe Flight Rewards r h =c h +k h min(z(t)-h min ,0)+k h min(h max -z(t),0) Among them, c h For safe flight rewards, k h is the height penalty weight, h min and h max They are the minimum and maximum flight altitudes respectively.

8. The machine learning training method for the UAV communication and perception integrated network according to claim 7, characterized in that: Drone Process Energy Bonus r e =k e (E r (t)-E r (t-dt)) Among them, k e is the energy penalty weight, and dt is the time step.

9. The machine learning training method for the UAV communication and perception integrated network according to claim 8, characterized in that: Drone flight guidance rewards Among them, ξ e is the critical energy value, d T,t-1 The distance between the drone and the perceived target in the last state sampling, d T,t The distance between the current state sampling drone and the sensing target, d d,t-1 The distance between the last state sampling drone and the end point of the position, d d,t The distance between the current state sampling drone and the end point of the position.

Images