Unmanned vehicle assisted information age minimization unmanned aerial vehicle path planning method

Abstract

The invention discloses an unmanned vehicle assisted information age minimization unmanned aerial vehicle path planning method, and belongs to the technical field of multi-unmanned aerial vehicle system tasks, and the method comprises the steps: constructing a perception scene, and constructing a state space, an action space, a state transfer function and a reward function in deep reinforcement learning based on the perception scene; obtaining the current state, obtaining the optimal action of the unmanned aerial vehicle based on the current state and an unmanned aerial vehicle flight path planning model, executing the optimal action of the unmanned aerial vehicle based on a charging module, storing experience, and repeatedly obtaining the optimal action of the current state until the maximum step number is reached or a preset sensing task is completed. And training an unmanned aerial vehicle flight path planning model based on the stored experience. The unmanned aerial vehicle is supplemented with energy in time through active movement of the unmanned vehicle, and the charging waiting time of the unmanned aerial vehicle is shortened. According to the method, the task completion rate and the data collection fairness can be maximized while the information age of the data is minimized in a data dynamic arrival scene.

Description

Technical Field

[0001] The present invention belongs to the technical field of multi-UAV system tasks, and in particular relates to an UAV path planning method with minimized information age assisted by an unmanned vehicle. Background Art

[0002] Unlike smart devices such as smartphones and tablets, drones have excellent environmental adaptability and flexible maneuverability, allowing them to quickly sense and collect data in complex environments. In addition, recent advances in drone commercialization have paved the way for the rapid adoption and deployment of low-cost, small drones that can be connected together and promote the formation of a new type of group perception, namely drone swarm perception. Drone swarm perception is applied in multiple fields, such as urban traffic management, application security enhancement, and environmental monitoring.

[0003] The information age of data is very important for applications such as urban traffic management and environmental monitoring, and directly affects their operational efficiency. The flight path of drones directly determines the information age of data. Many studies have used deep reinforcement learning methods to optimize the flight path of drones to reduce the information age of data. Although these works have achieved certain results in reducing the information age, since these works do not consider the dynamic arrival of data or use simple components to process the input of the intrinsic reward module, they have limited the effect of flight path optimization to a certain extent, and thus limited the effect of the model in reducing the information age of data.

[0004] In addition, the flight time of drones indirectly affects the information age of the data. There are a large number of long-duration tasks in smart city applications, which require drones to perceive and collect data on target objects for a long time. Due to energy limitations, drones cannot fly for a long time to collect data from mission points in a timely manner, which in turn affects the information age of the data. To solve the problem of drone energy limitations, some scholars have introduced static charging piles or mobile charging piles with fixed stations or mobile charging piles with non-fixed stations for static sensors.

[0005] Although static charging stations or mobile charging stations with fixed stations can alleviate the energy limitation problem of drones to a certain extent, these two types of charging stations cannot always replenish energy for drones in time because drones are constantly moving. In addition, mobile charging stations without fixed stations are aimed at static sensors and cannot effectively charge drones with high mobility.

[0006] In scenarios where data arrives dynamically, the performance of the drone path optimization method based on deep reinforcement learning in terms of the information age of the data is closely related to the feature extraction of the input. If a traditional convolutional neural network is used for feature extraction, the performance of the model will not be guaranteed. If a simple component (such as a linear layer) is used to process the input of the intrinsic reward module to extract features, local features cannot be captured, and the performance of the model cannot be strongly improved. Therefore, in scenarios where data arrives dynamically, it is challenging to design a deep reinforcement learning model to effectively extract input features to optimize the drone flight path and reduce the information age of the data.

[0007] In addition, in the UAV energy replenishment method based on charging piles, affected by the charging strategy and the location of the charging piles, the UAV often needs to fly a long distance and time to the fixed location of the static charging pile or the mobile charging pile, and cannot obtain energy in time, which increases the information age of the data to be collected. Therefore, how to replenish energy for the UAV in time to reduce the information age of the data is challenging. Summary of the invention

[0008] In order to solve the above technical problems, the present invention proposes an unmanned vehicle-assisted drone path planning method with minimized information age to solve the problems existing in the above-mentioned prior art.

[0009] To achieve the above object, the present invention provides a method for unmanned vehicle-assisted drone path planning with minimized information age, comprising:

[0010] Step 1: Construct a perception scene;

[0011] Step 2: construct a state space, action space, state transfer function and reward function in deep reinforcement learning based on the perception scene; and construct a UAV flight path planning model based on the state space, action space, state transfer function and reward function;

[0012] Step 3: Obtain the current state, obtain the best action of the drone based on the current state and the drone flight path planning model, execute the best action of the drone based on the charging module, and store the experience; repeat this step until the maximum number of steps is reached or the preset perception task is completed;

[0013] Step 4: Train the UAV flight path planning model based on the stored experience until the number of training times reaches a preset value; perform UAV path planning by deploying the trained UAV flight path planning model.

[0014] Optionally, the process of constructing the perception scene in step 1 includes:

[0015] Obtain the basic elements of the perception scene and determine the limiting conditions of the basic elements; obtain the drone behavior rules, initialize the drone and classify the drone energy state, set the probability and frequency of fixed-position points of interest generating data, and divide the perception process into several equivalent time slots; wherein the basic elements include drones, unmanned vehicles, fixed-position points of interest and obstacles.

[0016] Optionally, the state space includes two channels, one channel includes the positions of all obstacles and points of interest and the data volume of each point of interest at time t, and the other channel includes the horizontal coordinates of the unmanned vehicle and the drone at time t and the energy of each drone;

[0017] The action space of each drone includes abscissa increments, ordinate increments, data collection time, and charging time for flight.

[0018] Optionally, the process of constructing the reward function in step 2 includes: constructing a reward function based on task completion rate, data collection fairness and minimization of data information age, constructing a data collection fairness formula, transforming the reward function based on the data collection fairness formula, and obtaining the final reward function.

[0019] Optionally, the UAV flight path planning model in step 2 includes a model update module, an environment interaction module, an intrinsic reward module and a strategy optimization module; the model update module is used to update the intrinsic reward module and the strategy optimization module; the environment interaction module is used to interact with the environment according to the output of the strategy optimization module, and store the calculated gradient and interaction experience.

[0020] Optionally, the intrinsic reward module includes a target network and a prediction network; the strategy optimization module includes an action sampling network and a value evaluation network;

[0021] The action sampling network and the value assessment network are each composed of one linear layer, wherein the size of the hidden layer is obtained based on the number of drones;

[0022] The target network and the prediction network are both composed of pooling layers, convolutional layers and linear layers; wherein the Relu activation function is used between the convolutional layers.

[0023] Optionally, in the step 2, a total loss function is constructed, and the UAV flight path planning model is trained based on the total loss function, wherein the total loss function includes value loss, action loss, entropy loss and intrinsic reward loss; the value loss, action loss, entropy loss and intrinsic reward loss are obtained by the action sampling network, the value evaluation network, the target network and the prediction network.

[0024] Optionally, the process of executing the optimal action of the drone based on the charging module in step 3 includes:

[0025] The energy state of the drone is determined based on the energy of the drone in the current state. If the energy state of the drone is in a warning state, a rechargeable unmanned vehicle is searched. The rechargeable unmanned vehicle can reach the neighboring node of the corresponding drone on the map, and the distance between the neighboring node and the current position of the unmanned vehicle is not greater than the charging range.

[0026] The neighboring node is taken as the next possible position, and it is determined whether there is a reachable path between the unmanned vehicle and the next possible position. If not, the point closest to the next possible position is searched until a reachable point closest to the next possible position is found; after determining the next possible position, the position is updated.

[0027] Optionally, the location update process includes: determining whether the time required for the unmanned vehicle to travel from the current location to the next location exceeds a time slot. If so, deleting the last node in the original path and updating the next location to the last node in the original path; otherwise, calculating the available charging time for the unmanned vehicle at time t and ending the location update process.

[0028] Optionally, the UAV updates the charging time and performs charging actions according to the available charging time of the unmanned vehicle at time t; and calculates the data upload time according to the total time and time slot length used for flight, data collection and charging, and searches for the nearest unmanned vehicle to upload data; rewards are given according to the reward function, the current state is transformed into the next state according to the state transfer function, and the current state, action, reward, and next state are stored in the experience buffer.

[0029] Compared with the prior art, the present invention has the following advantages and technical effects:

[0030] The present invention first constructs a perception scene, and constructs a state space, action space, state transfer function and reward function in deep reinforcement learning based on the perception scene; constructs a UAV flight path planning model based on the state space, action space, state transfer function and reward function; the UAV path planning model includes an EfficientNet for extracting features and an intrinsic reward module that uses a convolutional neural network to process global information; then, the current state is obtained, the optimal action of the UAV is obtained based on the current state and the UAV flight path planning model, the optimal action of the UAV is executed based on the charging module, and the experience is stored; this step is repeated until the maximum number of steps is reached or the preset perception task is completed; the UAV flight path planning model is trained based on the stored experience until the number of training times reaches a preset value.

[0031] The present invention uses EfficientNet to effectively extract features in scenarios where data arrives dynamically, laying the foundation for the algorithm to obtain a good strategy. The present invention uses a convolutional neural network to quickly process the intrinsic reward module of the input representing global information, which can help drones explore a wider area. The unmanned vehicle-assisted drone energy replenishment method proposed in the present invention uses the active movement of the unmanned vehicle to replenish energy for the drone in a timely manner, and can also reduce the waiting time for drone charging. The present invention can minimize the information age of the data in scenarios where data arrives dynamically while maximizing the task completion rate and data collection fairness. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0033] Figure 1 A time-sensitive unmanned vehicle-assisted drone swarm intelligence perception scene diagram of an embodiment of the present invention;

[0034] Figure 2 The figure is an overall framework diagram of unmanned vehicle-assisted drone swarm intelligence perception according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0036] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] Embodiment 1

[0038] like Figure 1-2 As shown, in this embodiment, a method for drone path planning with minimized information age assisted by an unmanned vehicle is provided, comprising:

[0039] Step 1: Construct a perception scene;

[0040] In some specific implementations, the process of constructing the perception scene in step 1 includes:

[0041] Obtain the basic elements of the perception scene and determine the limiting conditions of the basic elements; obtain the drone behavior rules, initialize the drone and classify the drone energy state, set the probability and frequency of fixed-position points of interest generating data, and divide the perception process into several equivalent time slots; among them, the basic elements include drones, unmanned vehicles, fixed-position points of interest and obstacles.

[0042] Specifically, in step 1, the perception scene constructed includes a group of drones U, a group of unmanned vehicles G, some fixed points of interest P and some obstacles. The unmanned vehicle has both the functions of a mobile charging pile and a base station. The obstacles are all regular cuboids, and their heights are all above the height maintained by the drone. The scene has a fixed boundary that the drone cannot cross. The entire perception process is divided into T equivalent time slots of length τ. The possible actions of the drone in each time slot include: flying, data collection, charging and data uploading. The data is time-sensitive and arrives dynamically. Each point of interest has a probability of 1 / 2 to generate a data at each moment, that is, the data to be collected. The drone uploads the data to be collected to the base station, indicating that the task corresponding to the data is completed. Initially, the drones are scattered near the center of the mission area. The drones fly in the mission area to collect data from the points of interest. At the same time, they look for unmanned vehicles to charge and upload data. In this process, the drone will actively avoid obstacles to ensure flight safety. There are three states of drone energy: sufficient, warning and dangerous. When the drone energy is in the warning state, the drone will negotiate a location with the nearest unmanned vehicle for energy replenishment. When the drone's energy is in a dangerous state, the drone will fly to the nearest road and wait for nearby unmanned vehicles to recharge.

[0043] Step 2: Construct several elements of deep reinforcement learning methods based on perception scenarios, including: state space, action space, state transfer function and reward function; construct a UAV flight path planning model based on state space, action space, state transfer function and reward function;

[0044] In some specific embodiments, the state space includes two channels, one channel includes the positions of all obstacles and points of interest and the amount of data of each point of interest at time t, and the other channel includes the horizontal coordinates of the unmanned vehicle and the drone at time t and the energy of each drone;

[0045] The action space of each drone includes the abscissa increment for flight, the ordinate increment, the data collection time, and the charging time.

[0046] In some specific embodiments, the process of constructing a reward function in step 2 includes: constructing a reward function based on task completion rate, data collection fairness and minimization of data information age, constructing a data collection fairness formula, deforming the reward function based on the data collection fairness formula, and obtaining a final reward function.

[0047] Specifically, A) state space: Include Figure 1 There are two channels shown. One channel covers the positions of all obstacles and points of interest and the amount of data for each point of interest at time t, and the other channel includes the horizontal coordinates of the unmanned vehicle and drone at time t and the energy of each drone.

[0048] B) Action Space: For each drone u, is a four-tuple: in, and Indicates the increment of horizontal and vertical coordinates, and represent the data collection time and charging time, respectively.

[0049] C) State transfer function: Indicates that the state changes from s t Convert to t+1 The state of the system at time t is called s t When the drone performs a t After t According to the state transfer function Convert to s t+1 .

[0050] D) Reward Function: Refers to the current state of the drone u t Next, execute The instantaneous reward obtained after the task is completed. The reward function is designed according to the goal. When the drone performs time-sensitive tasks, the following goals need to be achieved: maximizing the task completion rate and data collection fairness while minimizing the information age of the data. Based on this goal, the reward function is as follows:

[0051]

[0052] in, and represents the time from the generation to the uploading of the uploaded data from the point of interest p and the uploaded data in the point of interest p in the time slot [t, t+1), where represents data collection fairness, which is constructed as follows:

[0053]

[0054] in, represents the amount of data collected at the point of interest p in the time slot [t, t+1). Since only the penalty for the collision of the drone is considered, the reward function is rewritten as:

[0055]

[0056] Among them, GR p,j UP p,j and X represent the generation time and upload time of the jth data from the point of interest p uploaded by the UAV in the time slot, as well as the duration of the entire perception process. and It represents the amount of data uploaded by UAV u from POI p in this time slot and the penalty it receives for collision.

[0057] In some specific embodiments, the UAV flight path planning model in step 2 includes a TRAINER-ACTION architecture, wherein TRAINER includes a model update module, and ACTION includes an environment interaction module, an intrinsic reward module (IRM) and a policy optimization module (PPO model); the model update module is used to update the intrinsic reward module and the policy optimization module; the environment interaction module is used to interact with the environment according to the output of the policy optimization module, and store the calculated gradient and interaction experience.

[0058] In some specific embodiments, the intrinsic reward module includes a target network (Target network) and a prediction network (Predictor network); the strategy optimization module includes an action sampling network (Actor network) and a value evaluation network (Critic network);

[0059] The action sampling network and the value assessment network are both composed of 1 linear layer, where the size of the hidden layer is obtained based on the number of drones;

[0060] Both the target network and the prediction network are composed of pooling layers, convolutional layers and linear layers; among them, Relu activation function is used between convolutional layers.

[0061] In some specific embodiments, a total loss function is constructed, and the UAV flight path planning model is trained based on the reward function and the total loss function, wherein the total loss function includes value loss, action loss, entropy loss, and intrinsic reward loss.

[0062] Specifically, to improve the efficiency of training, a TRAINER and an ACTION are designed, which are trained alternately. TRAINER is responsible for updating the PPO model including the Actor network and the Critic network, and the IRM including the Target network and the Predictor network. ACTION is mainly responsible for interacting with the environment, calculating gradients, and storing experience and gradients.

[0063] In this framework, we use the PPO model and IRM to enhance the stability of the system. Specifically, the PPO model contains an Actor network for sampling actions and a Critic network for evaluation. In order to effectively extract features in a dynamic environment, EfficientNet is used to extract features from the current state. See the EfficientNet structure for details. Figure 2 As shown in Figure 1, it includes three convolutional layers, a pooling layer, and a fully connected layer. The three convolutional layers include a convolutional layer with 32 channels, a convolutional layer with 1280 channels, and a mobile flip bottleneck convolutional layer. Considering the strong correlation between data upload and information in adjacent time periods, we use the data transmission rate of the drone at the previous moment ω t-1 , the proportion of uploaded data in the total amount of data collected ρ t-1 Combined with the obtained features. The combined features are converted into actions through the Actor network, and the batch states and actions are input into the Critic network to obtain value loss, action loss and entropy loss. The input of IRM is global information. We use CNN instead of linear layers to process the input of IRM, which is conducive to the drone exploring a wider area. IRM contains two networks-the Target network and the Predictor network. Its goal is to narrow the gap between the two networks as much as possible. We call this gap the intrinsic reward loss (IRM loss). The four losses are scaled and accumulated to obtain the total loss, and this loss is used to calculate the gradients of the parameters of the two networks in the PPO model and the Predictor network in IRM. TRAINER uses these gradients to update the PPO model and IRM using the gradient descent method.

[0064] like Figure 2 As shown in the figure, the Actor network and Critic network in the PPO model are both composed of 1 linear layer, and the size of the hidden layer is 256 + the number of drones * 2. The Target network and Predictor network in IRM are both composed of pooling layers, convolutional layers, and linear layers. Among them, the convolution kernel sizes of the pooling layer and convolutional layer are 2×2 and 3×3 respectively, the stride is 2, and the number of edge filling pixels is 0 and 1 respectively. The Relu activation function is used between convolutional layers. The size of the hidden layer is 256.

[0065] Step 3, ACTION obtains the current state, obtains the best action of the drone based on the current state and the drone flight path planning model, and the drone performs the best action based on the charging module. ACTION stores the experience.

[0066] In some specific implementations, the process of executing the optimal action of the drone based on the charging module in step 3 includes:

[0067] If the drone's energy status is in a warning state, search for a rechargeable unmanned vehicle that can reach the neighboring node of the corresponding drone on the map, and the distance between the neighboring node and the current location of the unmanned vehicle is not greater than the charging range;

[0068] Take the neighboring node as the next possible location, and determine whether there is a reachable path between the unmanned vehicle and the next possible location. If not, look for the point closest to the next possible location until a reachable point closest to the next possible location is found; after determining the next possible location, update the location.

[0069] In some specific implementations, the location update process includes: determining whether the time required for the unmanned vehicle to travel from the current location to the next location exceeds a time slot. If so, deleting the last node in the original path and updating the next location to the last node in the original path; otherwise, calculating the available charging time for the unmanned vehicle at time t and ending the location update process.

[0070] Specifically, the system determines whether there is a drone in the warning state at time t. If so, the drone negotiates with the nearest unmanned vehicle and moves to the negotiated position. After both parties move to the negotiated position, the drone in the warning state is charged. The remaining drones perform the flight actions in the best action, and all drones collect data. The energy status of the drone is checked again.

[0071] After the drone completes data collection in the time slot [t, t+1), the system records the drones in the area that are in a dangerous energy state and other drones. For drones in a dangerous energy state, the system searches for unmanned vehicles that can be charged for them in turn. Unmanned vehicles that can reach a node adjacent to a drone on the map and whose distance from the node to the current location of the unmanned vehicle is not greater than the charging range are called unmanned vehicles that can be charged. This node is the next possible location of the unmanned vehicle. Traverse the unmanned vehicles that have not yet found the next possible location. Their next possible location is the node on the map closest to the center point of the drone in the corresponding area. The determination of the next location of the unmanned vehicle requires judging whether there is a reachable path between its current location and the next possible location. If so, the unmanned vehicle preliminarily determines the next possible location as the next location; otherwise, find the node closest to the next possible location. If there is still no path between the node and the current location of the unmanned vehicle, continue to find the next closest point until a reachable point closest to the next possible location is found. After the next position is preliminarily determined, the position update process begins: determine whether the time required for the unmanned vehicle to travel from the current position to the next position exceeds a time slot. If so, delete the last node in the original path and update the next position to the last node in the original path; otherwise, calculate the available charging time for the unmanned vehicle at time t, and then exit the position update process.

[0072] To further prevent the drone from not getting energy replenishment for a long time, the system implements forced charging for drones that have run out of energy. If the distance between a drone that has run out of energy and a certain unmanned vehicle g is not greater than the charging range, the minimum value of the remaining time of the drone in this time slot and the available charging time of the unmanned vehicle at time t is calculated. To prevent the drone from wasting time charging when it is fully charged, the minimum value of the time required for full charging and the above value is taken as its charging time at time t. The drone updates the charging time and performs the charging action based on the available charging time of the unmanned vehicle at time t; and calculates the data upload time based on the total time for flight, data collection and charging and the time slot length, and searches for the nearest unmanned vehicle for data upload. The environment gives rewards according to the reward function, and the current state is transformed into the next state according to the state transfer function. ACTION stores the current state, action, reward, and next state in the experience buffer. Repeat this step until the maximum number of steps is reached or the preset perception task is completed.

[0073] Step 4: Train the UAV flight path planning model based on the stored experience until the number of training times reaches a preset value; perform UAV path planning by deploying the trained UAV flight path planning model.

[0074] The model training process is as follows:

[0075] ACTION takes batches of experience from the experience buffer. It uses the parameters of the Critic network in the PPO model to evaluate the s in these experiences.t and a t , and use the parameters of the IRM prediction network to evaluate the empirical s t+1 , thereby calculating the gradients of the Actor network and Critic network of the PPO model and the Predictor network of the IRM. The calculated gradients will be stored in two gradient buffers. TRAINER updates the PPO model and IRM. After TRAINER updates the two modules, ACTION copies the updated model parameters. Repeat step 3 and the model update process until the number of training times reaches the preset value.

[0076] The process of model deployment is as follows:

[0077] 1) Initialize the environment;

[0078] 2) The agent gets the current state s t , and use EfficientNet to process the current state to obtain state features, and combine the obtained state features with the relevant information of the previous moment to obtain new features. The agent uses the Actor network to generate actions, and the drone executes the actions.

[0079] 3) Determine whether the drones have reached the maximum number of steps or the perception tasks have been completed. If so, jump to step 4), otherwise t =s t+1 , and jump to step 2).

[0080] 4) The experiment is over.

[0081] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for unmanned vehicle-assisted drone path planning with minimized information age, characterized in that: The following steps are involved: Step 1: Construct a perception scene; Step 2: construct a state space, action space, state transfer function and reward function in deep reinforcement learning based on the perception scene; and construct a UAV flight path planning model based on the state space, action space, state transfer function and reward function; Step 3: Obtain the current state, obtain the best action of the drone based on the current state and the drone flight path planning model, execute the best action of the drone based on the charging module, and store the experience; repeat this step until the maximum number of steps is reached or the preset perception task is completed; Step 4: Train the UAV flight path planning model based on the stored experience until the number of training times reaches a preset value; perform UAV path planning by deploying the trained UAV flight path planning model.

2. The method for unmanned vehicle-assisted drone path planning with minimized information age according to claim 1, characterized in that: The process of constructing the perception scene in step 1 includes: Obtain the basic elements of the perception scene and determine the limiting conditions of the basic elements; obtain the drone behavior rules, initialize the drone and classify the drone energy state, set the probability and frequency of fixed-position points of interest generating data, and divide the perception process into several equivalent time slots; wherein the basic elements include drones, unmanned vehicles, fixed-position points of interest and obstacles.

3. The method for unmanned vehicle-assisted drone path planning with minimized information age according to claim 1, characterized in that: The state space includes two channels, one channel includes the positions of all obstacles and points of interest and the data volume of each point of interest at time t, and the other channel includes the horizontal coordinates of the unmanned vehicle and the drone at time t and the energy of each drone; The action space of each drone includes abscissa increments, ordinate increments, data collection time, and charging time for flight.

4. The method for unmanned vehicle-assisted drone path planning with minimized information age according to claim 1, characterized in that: The process of constructing the reward function in step 2 includes: constructing a reward function based on task completion rate, data collection fairness and minimization of data information age, constructing a data collection fairness formula, and transforming the reward function based on the data collection fairness formula to obtain the final reward function.

5. The method for unmanned vehicle-assisted drone path planning with minimized information age according to claim 1, characterized in that: The UAV flight path planning model in step 2 includes a model update module, an environment interaction module, an intrinsic reward module and a strategy optimization module; the model update module is used to update the intrinsic reward module and the strategy optimization module; the environment interaction module is used to interact with the environment according to the output of the strategy optimization module, and store the calculated gradient and interaction experience.

6. The method for unmanned vehicle-assisted drone path planning with minimized information age according to claim 5, characterized in that: The intrinsic reward module includes a target network and a prediction network; the strategy optimization module includes an action sampling network and a value evaluation network; The action sampling network and the value assessment network are each composed of one linear layer, wherein the size of the hidden layer is obtained based on the number of drones; The target network and the prediction network are both composed of pooling layers, convolutional layers and linear layers; wherein the Relu activation function is used between the convolutional layers.

7. The method for unmanned vehicle-assisted drone path planning with minimized information age according to claim 1, characterized in that: In the step 2, a total loss function is constructed, and the UAV flight path planning model is trained based on the total loss function, wherein the total loss function includes value loss, action loss, entropy loss and intrinsic reward loss; the value loss, action loss, entropy loss and intrinsic reward loss are obtained by the action sampling network, the value evaluation network, the target network and the prediction network.

8. The method for unmanned vehicle-assisted drone path planning with minimized information age according to claim 1, characterized in that: The process of executing the optimal action of the drone based on the charging module in step 3 includes: The energy state of the drone is determined based on the energy of the drone in the current state. If the energy state of the drone is in a warning state, a rechargeable unmanned vehicle is searched. The rechargeable unmanned vehicle can reach the neighboring node of the corresponding drone on the map, and the distance between the neighboring node and the current position of the unmanned vehicle is not greater than the charging range. The neighboring node is taken as the next possible position, and it is determined whether there is a reachable path between the unmanned vehicle and the next possible position. If not, the point closest to the next possible position is searched until a reachable point closest to the next possible position is found; after determining the next possible position, the position is updated.

9. The method for unmanned vehicle-assisted drone path planning with minimized information age according to claim 8, characterized in that: The location update process includes: determining whether the time required for the unmanned vehicle to go from the current location to the next location exceeds a time slot. If so, deleting the last node in the original path and updating the next location to the last node in the original path; otherwise, calculating the available charging time for the unmanned vehicle at time t and ending the location update process.

10. The method for unmanned vehicle-assisted drone path planning with minimized information age according to claim 8, characterized in that: The drone updates the charging time and performs the charging action according to the available charging time of the unmanned vehicle at time t; The data upload time is calculated based on the total time and time slot length used for flight, data collection and charging, and the nearest unmanned vehicle is searched for data upload; A reward is given according to the reward function, the current state is transformed into the next state according to the state transfer function, and the current state, action, reward, and next state are stored in the experience buffer.

Images