An unmanned aerial vehicle dynamic data acquisition method combined with data freshness optimization

Abstract

The present application relates to the technical field of unmanned aerial vehicle wireless communication and Internet of Things data acquisition, and specifically relates to a dynamic data acquisition method for unmanned aerial vehicles combined with data freshness optimization, which dynamically determines the hovering time of the unmanned aerial vehicle by introducing sensor transmission power adjustment and constructing an energy constraint model of flight energy consumption and hovering energy consumption. Through graph neural networks and disaster risk prediction, the present application realizes dynamic updating of node priority to ensure data acquisition of high-risk nodes. Further, the cross-attention mechanism is used to solve the state space dimension problem in the dynamic node scenario, ensuring stable operation of the reinforcement learning network. Through optimization of the hybrid action space and double-strategy updating, the system can efficiently handle discrete and continuous variables, thereby improving decision-making accuracy and task execution robustness. The method has significant advantages in disaster warning and environmental monitoring applications.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) wireless communication and Internet of Things (IoT) data acquisition technology, specifically to a UAV dynamic data acquisition method that incorporates data freshness optimization. Background Technology

[0002] In emergency communication scenarios where communication nodes are interrupted due to remote signal blind spots or unforeseen circumstances, drones are often used as data acquisition relay nodes due to their high flexibility and mobility. In wireless sensor network applications with extremely high real-time requirements, such as environmental monitoring and natural disaster early warning, the system is highly sensitive to the timeliness of information.

[0003] Traditional wireless sensor network research typically uses network latency as an indicator of information real-time performance, but this indicator fails to fully reflect the up-to-date state of the data collected by the target node. Therefore, the academic community has introduced the concept of Age of Information (AoI), defined as the cumulative time between the moment information is generated at the source and the moment it is successfully received by the receiver, used to accurately measure the timeliness of information.

[0004] Existing technologies have significant shortcomings in handling UAV-assisted data acquisition. First, current path planning methods often only optimize discrete hovering points or flight trajectories in continuous space, lacking the ability to jointly optimize mixed action spaces. Second, in the field of deep reinforcement learning, conventional network structures cannot simultaneously and efficiently output discrete and continuous actions of different dimensions, easily leading to dimensionality explosion in the state space. Existing research often uses peak data freshness as the optimization objective, neglecting the weighted average data freshness based on preset node priorities, and failing to consider sensor transmission power as a variable dynamically determining UAV hovering time. This results in a lack of refined energy consumption and freshness optimization mechanisms, making it impossible to achieve optimal data acquisition under strict total power constraints.

[0005] Furthermore, existing technologies still face two major systemic bottlenecks when dealing with highly dynamic disaster scenarios: First, there is a lack of prior awareness of the evolution of the physical environment. Existing systems rely heavily on static preset weights to assess the importance of nodes, and cannot predict the spread of disasters based on real three-dimensional environmental dynamic parameters. This results in drones being unable to collect critical early warning data before nodes are damaged by disasters.

[0006] Secondly, it is incompatible with sudden tasks and lacks flexibility in total energy constraints. The input feature dimension of traditional deep reinforcement learning algorithms is fixed. When a new task node is suddenly added to the monitoring area, it will cause a mismatch in the state space dimension, which will lead to network collapse. Furthermore, under the strict total energy constraints of the system, drones are often unable to go to sudden critical nodes for emergency data collection due to unplanned power shortages.

[0007] Therefore, there is an urgent need for a highly energy-efficient method that can effectively optimize the sensor node access sequence, UAV flight speed, and sensor transmission power under the constraint of limited total energy, and minimize the weighted average data freshness of the system based on a special neural network architecture. Summary of the Invention

[0008] The purpose of this invention is to provide a UAV dynamic data acquisition method that combines data freshness optimization to solve the above-mentioned technical problems. This method enables dynamic adjustment of sensor node acquisition priority, early acquisition of key early warning data from high-risk nodes, avoidance of data loss due to node damage, and accurate prediction of disaster evolution trends. It also prioritizes data acquisition from high-risk nodes. Furthermore, it solves the state space dimension problem in dynamic node scenarios, ensuring the stable operation of reinforcement learning networks and improving the system's decision-making accuracy and robustness.

[0009] The objective of this invention can be achieved through the following technical solutions: A method for dynamic data acquisition from unmanned aerial vehicles (UAVs) that incorporates data freshness optimization, the method comprising: S1. Dynamically acquire the set of sensor nodes and the status of the UAV; acquire the multi-source heterogeneous environment characteristics of each sensor node, and construct a directed graph topology based on the multi-source heterogeneous environment characteristics, and output the adjacency matrix; S2. Input the adjacency matrix and multi-source heterogeneous environmental features into the graph neural network, output the disaster risk probability of each sensor node, and map the node priority according to the disaster risk probability to output the dynamic urgency weight of each sensor node. S3. Construct an energy constraint model and an optimization model for node states; the energy constraint model includes flight energy consumption and hovering energy consumption; the node state includes data freshness, which is an amount that increases over time and is updated when data is uploaded, and a weighted average data freshness is calculated based on dynamic urgency weights. S4. Construct a decision-making process model, and use a cross-attention mechanism to fuse features of the UAV state and node state, and output a state vector with fixed dimensions. S5. Solve the decision process model and construct a strategy model for outputting hybrid action vectors; S6. Redistribute energy and execute strategies based on the hybrid action vector.

[0010] Furthermore, the method in step S1 includes: S11. Let the sensor node set be... , of which Location coordinates of each sensor node For a three-dimensional fixed vector; in the first... At a time step, when received When a sudden alarm node requests access, the total number of nodes in the current sensor node set is updated. The updated sensor node set is: ; S12, using drones in the The drone's position is updated based on the flight direction angle and flight speed at each time step; S13. Establish a free-space line-of-sight communication link model between the UAV and sensor nodes; calculate the communication link between the UAV and the sensor nodes while hovering. The three-dimensional spatial distance between sensor nodes; S14. Calculate the communication signal gain based on the channel gain per unit distance; and calculate the UAV's influence on the first [unclear - likely a specific data point or parameter] based on the communication bandwidth, sensor transmission power, Gaussian white noise power, and communication signal gain. The data transmission rate when each sensor node collects data.

[0011] Furthermore, the method in step S1 includes: S15. Multi-source heterogeneous environmental features include static geographical prior features and real-time meteorological dynamic features. Static geographical prior features and real-time meteorological dynamic features are spliced ​​together to form the environmental perception feature vector of sensor nodes. S16. Construct a dynamic directed graph topology based on environmental perception feature vectors. ;in, Represents a set of nodes, corresponding to the first... A set of sensor nodes at each time step ; This represents a set of edges, corresponding to the directed connections between nodes; Representing the adjacency matrix; calculating the elements of the adjacency matrix based on the physical laws of thermal convection. The adjacency matrix elements are nodes. To the node The weight of the directed edge.

[0012] Furthermore, the graph neural network includes a spatial feature aggregation layer and a temporal feature evolution layer connected in series. The method in step S2 includes: S21. Summarize the environmental perception feature vectors of all sensor nodes into a full node environmental perception feature matrix. Use graph convolution operation to weighted aggregate the adjacency matrix and the full node environmental perception feature matrix to output the spatial latent feature matrix. S22. In the temporal feature evolution layer, the continuous The spatial latent feature matrix of each time step is input into the gated recurrent unit; the gated recurrent unit includes a reset gate and an update gate; the reset gate is calculated based on the temporal feature vector of the previous time step and the spatial feature vector of the current time step, and the update gate is calculated based on the temporal feature vector of the previous time step and the spatial feature vector of the current time step; the temporal feature vector of the previous time step is modulated based on the reset gate, and candidate temporal feature vectors are calculated by combining it with the spatial feature vector of the current time step; the temporal feature vector of the previous time step and the candidate temporal feature vectors are weighted and combined based on the update gate to output the temporal latent feature vector of the current time step; S23. Input the temporal latent feature vector into the multilayer perceptron and output the disaster risk probability of the sensor nodes within the target time window; S24. Based on the disaster risk probability, the static priority weights of each node are reconstructed into dynamic urgency weights through an exponential mapping function; the static priority weights are determined during the system initialization phase.

[0013] Furthermore, the method in step S3 includes: S31. Calculate the drone's propulsion power based on the drone's status and fuselage parameters; S32. Calculate flight energy consumption based on the flight time of the nth time step, calculate the hovering data collection time of the UAV based on the amount of data sent by the sensor node and the transmission rate, and calculate hovering energy consumption based on the hovering data collection time. S33. Construct a total available energy constraint for the system, limiting the sum of the cumulative flight energy consumption and hovering energy consumption of the UAV during the mission cycle to not exceed the preset energy limit; S34. Define the data freshness of each sensor node. When the data upload of the sensor node is completed, update the corresponding data freshness to the initial value. When the data is not uploaded, increase the data freshness over time and do not exceed the preset upper limit. S35. Calculate the weighted average data freshness of the system based on the data freshness of each sensor node and the corresponding dynamic urgency weight.

[0014] Furthermore, the decision process model is a Markov decision process model, and the method in step S4 includes: S41. Convert the UAV state into an ontology state vector. The node state also includes dynamic urgency weight, disaster risk probability, and node maximum transmission power. Convert the node state into a node state vector. Map the ontology state vector to a query vector. Map each node state vector to a key vector and a value vector. S42. Calculate the attention score of the UAV’s current state to the sensor node by dot product attention, and obtain the attention weight after normalizing the attention score. S43. Use attention weights to perform a weighted summation of the value vectors of all nodes, and output a global environment feature with constant dimension. S44. Concatenate the ontology state vector with the global environment features to form a state vector with fixed dimensions.

[0015] Furthermore, step S4 also includes: S45. Construct a hybrid action space, which includes discrete actions for representing node access selection and continuous actions for representing flight speed and transmission power; wherein, the action at each time step is a combination of node access actions, flight speed control parameters, and transmission power control parameters. S46. Define a state transition function to represent the probability distribution of transitioning from the current state to the next state after performing an action; S47. Define a reward function, which includes a term based on weighted average data freshness, a penalty term for data freshness exceeding a preset upper limit, a penalty term for energy consumption, and a reward term for high-risk nodes completing data upload.

[0016] Furthermore, step S5 includes: S51. The policy model constructed by the proximal policy optimization algorithm is modeled as the probability distribution of choosing actions under a given state. S52. Output discrete and continuous policies respectively through a dual-policy network; the discrete policy outputs the specific probability distribution of discrete actions, and the continuous policy outputs the Gaussian distribution of continuous actions by updating the action mean and action standard deviation. S53. Define the policy-based expected reward as the reward function, and update the dual-policy network using the policy gradient method; S54. Update the weights of the dual-policy network using the gradient ascent method.

[0017] Furthermore, step S5 also includes: S55. Calculate the ratio of the action probability of the old and new strategies for both discrete and continuous strategies. S56. Introduce a value network to estimate state value and calculate temporal difference error; calculate temporal difference error based on the reward of the current time step and the state value of the next time step. S57. Based on the time-series difference error, the errors of multiple time steps are weighted and summed according to a preset discount factor to obtain an estimate used to characterize the value of the action. S58. Set the shear range parameters and construct a policy objective function that includes the shear function based on the action probability ratio and the estimate. S59. Update the network parameters of the discrete policy and the continuous policy respectively using the gradient ascent method; S510. Construct a value network loss function to characterize the difference between the predicted state value and the target reward, and update the value network parameters based on the value network loss function; introduce a policy entropy function, and construct a total loss function together with the policy objective function and the value network loss function to update the network parameters.

[0018] Furthermore, step S6 includes: S61. Before executing the hybrid action space, if the current state contains high-risk nodes and the disaster risk probability of the sensor node exceeds a preset threshold, then calculate the dynamic risk redistribution of the sensor node. S62. Based on the disaster risk probability and preset threshold, the remaining sensor nodes to be accessed are divided into a safe subset and a critical subset; S63. Send a transmission power adjustment command to all sensor nodes in the safety subset to set the transmission power to the maximum hardware transmission power and reduce hovering energy consumption. S64. When the reduced hovering energy consumption is greater than or equal to the flight energy consumption required to perform the action of accessing the target sensor node, the saved hovering power is allocated to the flight energy consumption to fly to the target high-risk node, and the optimal action strategy output in step S5 is executed; when the reduced hovering energy consumption is less than the flight energy consumption required to perform the action of accessing the target sensor node, the sensor nodes of the safe subset with the lowest score are eliminated in ascending order according to the attention weight calculated in step S4, and the reduced hovering energy consumption is greater than or equal to the flight energy consumption required to perform the action of accessing the target sensor node.

[0019] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention uses the system-weighted average data freshness based on preset node priority weights as the optimization objective. In particular, it innovatively introduces sensor transmission power as a continuous control variable, dynamically adjusting the hovering time of the UAV at each data collection point, thereby constructing a total available energy constraint mechanism consisting of flight energy consumption and dynamic hovering energy consumption. This mechanism ensures the optimization of core node data freshness and resource allocation under limited energy conditions.

[0020] 2. This invention achieves accurate prediction of disaster spread trends by constructing a dynamic directed graph topology coupled with wind direction and micro-topography, combined with a spatiotemporal graph convolutional network. Based on the calculated disaster risk probability, the node priority weights are dynamically updated, enabling the system to prioritize the data freshness of high-risk nodes and prevent the loss of critical early warning data before node damage.

[0021] 3. This invention achieves a lossless mapping from the states of a variable number of nodes to features of a fixed dimension by introducing a cross-attention mechanism. This mechanism ensures that the reinforcement learning network can still operate stably in scenarios with sudden node intervention or dynamic changes in the number of nodes, avoiding the problems of state space dimension explosion and network collapse, thereby significantly improving the system's environmental adaptability.

[0022] 4. The energy dynamic redistribution mechanism proposed in this invention reduces the hovering energy consumption of the UAV by adjusting the transmission power of the safe subset nodes, thereby achieving cross-domain replacement of hovering energy consumption and flight energy consumption. This mechanism ensures that the system can still complete the data acquisition task of sudden high-risk nodes under the condition of limited total energy, further improving the system's energy utilization efficiency and mission robustness.

[0023] 5. This invention, by jointly optimizing the hybrid motion space, differs from existing technologies that simply optimize discrete hovering points or flight trajectories in continuous space. This method utilizes independent discrete and continuous strategies to simultaneously and efficiently handle discrete variables, including sensor node access order, and continuous variables, including flight speed and transmission power, greatly expanding the boundaries of decision-making optimization and improving control accuracy.

[0024] 6. To support the optimization of hybrid action spaces, this invention innovatively designs a policy network in the proximal policy optimization algorithm architecture that includes a shared feature extraction layer and a branch structure. After improving feature reuse efficiency through bottom-level shared feature extraction, the branch structure outputs discrete and continuous actions respectively. These two branches each have independent objective functions and are updated separately, thereby solving the problems of state space dimensionality explosion and gradient conflict when jointly outputting action spaces of different dimensions. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of a scene model for the UAV-assisted data acquisition system of the present invention; Figure 2 This is a diagram illustrating changes in data freshness (AoI). Figure 3 The simulation results of the near-end policy optimization (PPO) algorithm designed for the special network structure of this invention are shown in the figure. Detailed Implementation

[0026] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0027] like Figure 1 and Figure 2 The method shown is a UAV dynamic data acquisition method that incorporates data freshness optimization. The method includes: S1. Dynamically acquire the set of sensor nodes and the state of the UAV; acquire the multi-source heterogeneous environment features of each sensor node, and construct a directed graph topology based on the multi-source heterogeneous environment features, outputting the adjacency matrix; where the adjacency matrix is ​​used to characterize the influence relationship between nodes, and the multi-source heterogeneous environment features are used to describe the state of the nodes themselves; this step realizes the joint modeling of node attributes and node relationships, providing the input basis for the subsequent graph neural network.

[0028] S2. Input the adjacency matrix and multi-source heterogeneous environmental features into the graph neural network, output the disaster risk probability of each sensor node, and map the node priority according to the disaster risk probability to output the dynamic urgency weight of each sensor node; the disaster risk probability is used to characterize the state change trend of the node; the dynamic urgency weight is used to characterize the relative importance of the node in the scheduling; the environmental information is transformed into a weight expression form that can be used for optimization decision-making.

[0029] S3. Construct an energy constraint model and an optimization model for node states; the energy constraint model includes flight energy consumption and hovering energy consumption; the node state includes data freshness, which is an amount that increases over time and is updated when data is uploaded, and a weighted average data freshness is calculated based on dynamic urgency weights; the energy model is used to constrain UAV behavior; data freshness is used to describe the timeliness of information; step S3 is used to achieve coupled modeling of constraints and optimization objectives.

[0030] S4. Construct a decision process model, and use a cross-attention mechanism to fuse features of UAV state and node state, outputting a state vector with fixed dimensions; unify the expression of UAV state and node state, and use a cross-attention mechanism to weighted aggregate information from multiple nodes, mapping variable-length node information into a fixed-dimensional state vector, which solves the problem of inconsistent state dimensions caused by changes in the number of nodes.

[0031] S5. Solve the decision process model and construct a policy model for outputting state vectors; this model is used to implement the decision mapping process from the state space to the action space. S6. Based on the state vectors of the sensor nodes, energy is redistributed and strategies are executed. Under energy-constrained conditions, hovering energy consumption is altered by adjusting transmission power, and nodes are screened or adjusted as necessary to ensure the execution strategy meets energy constraints.

[0032] Specifically, a set of sensor nodes was originally deployed on a two-dimensional plane. , of which Location coordinates of each sensor node It is a three-dimensional fixed vector. In the... At each time step, the system receives... The system dynamically updates the total number of nodes to reflect the incoming access requests from each emergency alarm node. The updated full set of nodes is The drone in the Position coordinates at each time step Includes horizontal coordinates and constant flight altitude Define drones in the first Flight direction angle at each time step With flight speed The drone's location is updated accordingly.

[0033] Establish a free-space line-of-sight communication link model between the UAV and sensor nodes. Based on the free-space path loss model, the UAV communicates with the sensor nodes while hovering. Three-dimensional spatial distance between sensor nodes Calculate as follows: Based on unit distance channel gain Calculate the gain of the communication signal As shown in the following formula: Based on Shannon's formula, using communication bandwidth Sensor transmission power and Gaussian white noise power Calculate the UAV for the first The data transmission rate when each sensor node collects data As shown in the following formula: Specifically, while performing data acquisition, the UAV acquires multi-source heterogeneous environmental characteristics from each sensor node, including static geographical prior features and real-time meteorological dynamic features. The static geographical prior features include node elevation... Geological combustible index Real-time meteorological dynamics include node ambient temperature. relative humidity Smoke concentration and rate of temperature rise The above features are concatenated into nodes. Environmental perception feature vector The expression is: To capture the physical evolution trend of disasters, a dynamic directed graph topology coupled with wind direction and terrain is constructed on the airborne terminal. The set of nodes Corresponding to the The complete set of nodes at each time step edge set The directed connection relationships between corresponding nodes, adjacency matrix elements For nodes To the node The directed edge weights, calculated based on the physical laws of thermal convection, are as follows: in, For the first Environmental wind speed vector at each time step For nodes Pointing to node The direction unit vectors, and their dot product, represent the effective propulsive kinetic energy of the wind field; For nodes With nodes The elevation difference characterizes the terrain slope effect; The truncation distance is used to maintain the sparsity of the graph matrix; , These are configurable weighting coefficients.

[0034] Specifically, the dynamic directed graph topological adjacency matrix constructed in step 1... With the full set of node environment perception feature matrix The input is a lightweight Spatio-Temporal Graph Convolutional Network (ST-GCN) deployed on an airborne device. The network consists of a spatial feature aggregation layer and a temporal feature evolution layer connected in series.

[0035] First, in the spatial feature aggregation layer, the adjacency matrix... Self-loop completion and degree normalization are performed; the calculation formula is as follows: In the formula, An identity matrix whose dimensions match the total number of nodes. Let be a degree matrix, and let its diagonal elements be the sum of the weights of the corresponding nodes.

[0036] By employing graph convolution operations based on first-order Chebyshev polynomial approximation, spatial feature interaction fusion is achieved, outputting a spatial latent feature matrix. The calculation formula is: in, is a learnable linear dimensionality-reduced weight matrix, and ReLU is the modified linear unit activation function.

[0037] Subsequently, in the temporal feature evolution layer, the continuous The spatial latent feature sequence at each time step is input into a gated recurrent unit (GRU) to capture the temporal evolution of environmental features. For any node... , its first The spatial feature vectors at each time step are The iterative calculation process of GRU is as follows: In the formula, To reset the door, To update the door, It is the Sigmoid activation function. The hyperbolic tangent activation function is used. For Hadama accumulation, , , The weight matrix is ​​a learnable matrix. For nodes The output temporal latent feature vector.

[0038] The temporal latent feature vector output by the GRU is input into a multilayer perceptron, mapped to the 0-1 interval by the Sigmoid activation function, and the output is the node within the target time window. Disaster risk probability The disaster risk probability vector of all nodes is .

[0039] Based on the probability of disaster risk, the static priority weights of each node are assigned using an exponential mapping function. Reconstructed into dynamic urgency weights The calculation formula is: in, , This is a configurable scaling factor.

[0040] Specifically, the total energy consumption of a drone is broken down into flight energy consumption. With hovering energy consumption Based on the drone's flight speed Combined with hovering blade profile power Hovering Induction Power Average induction velocity Blade tip velocity air density Airframe drag ratio Rotor structural stiffness and rotor disk area Calculate the propulsion power of the UAV As shown in the following formula: According to the Flight time at each time step Calculate flight energy consumption Based on the amount of data sent by the sensor nodes With transmission rate Calculate the hovering time of the drone for data collection. Calculate hovering energy consumption .

[0041] Establish the total available energy of the system Constraints require that the cumulative energy consumption meet the following requirements. .

[0042] Define the data freshness of each sensor node When data is successfully uploaded, the value is reset to 0. When no data is uploaded, the value increases linearly with time step, but the maximum freshness level does not exceed the system's tolerable limit. Based on the dynamic urgency weights output in step 2. With the current data freshness of the node Calculate the dynamic weighted average data freshness of the system The formula is as follows: Specifically, a hybrid action space collaborative optimization problem is established with the objective of minimizing the dynamic weighted average data freshness throughout the entire task cycle of the system, and this problem is transformed into a Markov Decision Process (MDP) model.

[0043] To address the state space dimensionality mismatch caused by sudden node intervention, a cross-attention mechanism is introduced before the state input reinforcement learning network to map variable-length states to fixed-length features. The UAV body state vector is defined as follows. ,in For the first Remaining energy of the drone at each time step; define nodes. Node state vector ,in For nodes The maximum transmission power. The calculation process of the cross-attention mechanism is as follows: Introduce three independent linear weight matrices , , Mapping the UAV's state vector to a query vector Mapping the state vectors of each node to key vectors AND value vector The calculation formula is: Calculate the current state of the drone for each node using dot product attention. Attention Score Attention weights are obtained after normalization using the Softmax function. The calculation formula is: in, For attention feature space dimension, This is a dimension scaling factor used to prevent gradient vanishing.

[0044] By using attention weights to perform a weighted summation of the value vectors of all nodes, a global environment feature with constant dimensionality is output. The calculation formula is: The UAV's own state vector is concatenated with global environmental features to form a state vector with fixed dimensions. Complete the state space construction of the Markov decision process model, expressed as: Specifically, setting up a hybrid motion space , where discrete sets Represents the set of visited nodes, a contiguous set. Represents the set of flight speeds, a continuous set Represents the set of transmitted power. The mixed action vector at each time step Includes discrete node access actions Continuous flight speed control maneuvers and sensor power control action The expression is: Define the state transition function Defined as executing a hybrid action vector Then transition to state probability distribution .

[0045] Set reward function Define instant rewards This is the negative value of the system's dynamically weighted average data freshness at the current time step, with an additional penalty for exceeding the tolerance limit for data freshness. Energy depletion penalty item Rewards for completing data collection at high-risk nodes The formula is expressed as: Among them, when When the node completes data upload, trigger Positive rewards This is the risk probability threshold.

[0046] Specifically, the Proximal Policy Optimization (PPO) algorithm is used to solve the Markov decision process. To adapt to the hybrid action space, the policy network adopts a shared feature extraction layer plus a branching structure. The discrete action branch outputs the node visit probability distribution, while the continuous action branch outputs the Gaussian distribution parameters of flight speed and transmission power. The discrete and continuous policies have independent objective functions and are updated separately. The value network adopts an independent branching structure and outputs a scalar state value evaluation.

[0047] strategy Modeling as a given state Select action The probability distribution, and the network parameters are The formula is expressed as follows: Specifically, a dual-policy network is used to output discrete policies respectively. With continuous strategy Discrete Strategy Output the specific probability distribution of discrete actions, and the continuous strategy. By updating the action mean With the standard deviation of movement To output a Gaussian distribution of continuous actions. Discrete strategy. With continuous strategy Each has its own independent objective function and And it will be updated independently.

[0048] Define policy-based expected return As shown in the following formula, where The trajectory of the intelligent agent's interaction with the environment. The reward for the trajectory: The parameters of the neural network are updated using the policy gradient method, and the gradient of the loss function is defined. As shown in the following formula Updating neural network parameters using gradient ascent method Set the learning rate The formula is expressed as follows: To limit the range of policy parameter updates and ensure training stability, the ratio of action probabilities between the old and new policies is calculated for both discrete and continuous policies. As shown in the following formula: Introducing value networks Predicting state value and calculating time series difference error As shown in the following formula, where Discount factor: Based on the generalized advantage estimation, the relative value of a specific action is quantified, and a weighting adjustment coefficient is introduced. Calculate the advantage function As shown in the following formula: Set the shear range parameters By combining policy improvement and update magnitude constraints, a near-end policy optimization objective function is constructed. As shown in the following formula: Maximize the objective function of the discrete strategy using the gradient ascent algorithm. With continuous policy objective function Update the corresponding strategy network parameters and output the final optimized discrete node access order and continuous flight control parameters.

[0049] Define the value network loss function The formula is the mean squared error between the predicted value and the target return. Define the policy entropy function. Used to encourage exploration. Combined to construct the total loss function. The formula is as follows: Specifically, in executing hybrid action vectors Previously, if the decision involved flying to a sudden high-risk node, the system would assess the additional expected flight energy consumption to that node. If the energy level exceeds the current remaining energy budget, a dynamic energy reallocation mechanism will be triggered.

[0050] Based on disaster risk probability Risk probability threshold Divide the remaining nodes to be accessed into a safe subset. With the critical subset The division rule is: when At that time, the node is assigned to the safe subset. The remaining nodes are assigned to the critical subset. .

[0051] The system issues a transmission power adjustment command to all nodes within the safe subset, setting their transmission power to the maximum hardware transmission power. Based on Shannon's channel capacity formula and hovering energy consumption model, increasing transmission power will shorten data transmission time, thereby reducing drone hovering energy consumption and squeezing out the saved hovering power. The calculation formula is: like If so, the saved hovering power will be allocated to the flight energy consumption for flying to the target high-risk node, and the optimal action strategy output by the near-end strategy optimization algorithm will be executed; if Then, based on the attention weights calculated in step S4 The lowest-scoring safe subset of nodes is removed in ascending order until the released energy redundancy meets the flight requirements.

[0052] To make the objectives and technical solutions of this invention clearer, the following embodiments are provided: I. System Parameter Initialization and Environment Setup: Deploy a two-dimensional target region with a side length of 1000m, with an initial total number of sensor nodes of [number missing]. Each sensor node is assigned four different levels of basic priority weights. The drone maintains a constant flight altitude. Set to 100m, maximum flight speed The speed is set to 30 m / s. Maximum transmission power limit of the sensor. Set to 1W, communication bandwidth Set to 1MHz, ambient Gaussian white noise power Set to -110dBm, channel gain per unit distance Set to -60dB.

[0053] UAV aerodynamic parameter initialization: hovering blade profile power Set to 100W, hovering guidance power Set to 120W, blade tip speed The average induction velocity is set to 120 m / s. Set at 4.03 m / s, air density Set to 1.23 kg / m 3Airframe drag ratio The rotor structural stiffness is set to 0.48. Set to 0.05, rotor disk area Set to 0.50m 2 .

[0054] Reinforcement learning training parameter initialization: learning rate set to 3e-4, discount factor... The generalized dominance estimation parameter is set to 0.99. Set to 0.95, BatchSize to 64, and the strategy pruning range. Set the value to 0.2 and the total number of training rounds to 2000.

[0055] II. Dynamic Graph Construction and Implementation of Disaster Spatiotemporal Prediction At each time step The system first acquires multi-source heterogeneous environmental perception data from all nodes, and then concatenates the data to generate environmental perception feature vectors for each node. The feature dimension is 6. Simultaneously acquire current environmental wind speed vector and node elevation data, construct a dynamic directed graph topology coupling wind direction and micro-topography, generating a dimension of... adjacency matrix .

[0056] Adjacency matrix With characteristic matrix The input is an airborne ST-GCN model, which is deployed on the UAV's airborne edge computing unit. The specific network structure is as follows. The spatial graph convolutional layer has an input dimension of 6 and an output dimension of 32. It uses a first-order Chebyshev polynomial approximation convolution kernel and the activation function ReLU to complete spatial feature extraction. The temporal GRU layer has an input dimension of 32 and a hidden layer dimension of 64. It adopts a single-gated recurrent unit structure to complete temporal feature extraction. The output layer consists of two fully connected layers with a hidden layer dimension of 32. The terminal uses the Sigmoid activation function to output the disaster risk probability of each node. .

[0057] After the model outputs the risk probability, the dynamic urgency weight of each node is calculated according to the dynamic weight mapping formula. The system's weighted average data freshness calculation model is updated synchronously.

[0058] III. Implementation of State Feature Aggregation Based on Cross-Attention Mechanism At the current time step The intelligent agent obtains the drone's own state vector. and the composite state vector of all nodes. Then input it into the cross-attention aggregation module. The specific implementation process of the module is as follows: Linear mapping layer: Employs 3 independent fully connected layers, each corresponding to a weight matrix. , , With input dimensions of 6 and 10 respectively and output dimensions of 128, the drone's body state and node composite state are mapped to a unified attention feature space. The attention weight calculation layer calculates the attention score through dot product attention, normalizes it using the LeakyReLU activation function and the Softmax function, and outputs the attention weights of each node. ; The feature aggregation layer performs a weighted summation of the value vectors of all nodes based on attention weights, and outputs a global environment feature with a fixed dimension of 128. ; The state concatenation layer concatenates the 6-dimensional UAV body state vector with the 128-dimensional global environment features to generate a fixed-dimensional 134-dimensional state vector. Input the subsequent PPO network.

[0059] IV. Strategy Model Construction To achieve collaborative optimization of the hybrid action space, this embodiment employs independent yet collaborative topology design for the policy network (Actor) and the value network (Critic). The network structure is as follows: Shared feature extraction layer: Receives a fixed-dimensional state vector output by the cross-attention module. It is constructed using 3 fully connected layers, with each hidden layer having a neuron dimension of 1024, and uses the ReLU activation function to extract high-order state features of the environment; Discrete action branches: corresponding discrete policy network parameters It receives high-order state features from the shared feature extraction layer, maps them through two fully connected layers, and then normalizes them using the Softmax function. The output is a discrete probability distribution vector with dimensions equal to the upper limit of the total number of nodes, used to select the optimal node access action. This branch has its own independent discrete objective function. ; Continuous action branches: corresponding continuous policy network parameters The system receives high-order state features from the shared feature extraction layer in parallel, and after mapping through two independent fully connected layers, outputs the action mean of continuous actions. With the standard deviation of movement Specifically, it includes the average flight speed of the drone. with standard deviation Average value of sensor transmission power with standard deviation This is used to construct a Gaussian distribution for continuous actions, and this branch has an independent continuous objective function. ; The Critic value network employs an independent branch architecture to evaluate state value. It consists of four fully connected layers with hidden layer neurons having dimensions of 1024, 1024, 512, and 1 respectively. The terminals utilize the Tanh activation function and undergo LayerNorm normalization. The output is a scalar state value assessment. .

[0060] V. Algorithm Training Loop and Network Parameter Update Process The training cycle steps are as follows: Step 1: The agent interacts with the environment to set the step size. The environment receives the hybrid action vector. Calculate dynamic hovering time based on energy consumption model The increase in energy consumption is used to determine whether it violates the total available energy. Constraints, update the next state Synchronously update the dynamic graph topology and node risk probabilities to complete cross-attention feature aggregation.

[0061] Step 2: Based on the reward function formula Calculate and store instant rewards Will include state Hybrid Action Vectors ,award and state value The transfer tuple is stored in the trajectory buffer.

[0062] Step 3: Using the trajectory buffer data, according to the formula... Calculate the timing difference error and follow the formula. Calculate the generalized dominance estimate for all time steps. Simultaneously, calculate the standardized dominance value to reduce variance; the formula is as follows: .

[0063] Step 4: Targeting Discrete Strategies With continuous strategy Calculate the probability ratio of the old and new mixed strategies for each. We construct the proximal policy optimization objective loss function for both discrete and continuous branches. .

[0064] Step 5: Calculate the value network loss function The formula is the mean squared error between the predicted value and the target return. Calculate the policy entropy function. Used to encourage exploration. Combined to construct the total loss function. The formula is as follows: Step 6: Minimize the negative of the total loss function using the gradient descent algorithm, and synchronously update the discrete policy network parameters according to their respective independent objective functions. Continuous policy network parameters With value network parameters .

[0065] Step 7: Adjust exploration strategy; if the current round accumulates rewards... Greater than the historical best reward This increases the standard deviation of the exploratory actions in continuous movements. Otherwise, based on the attenuation coefficient Attenuation is achieved using the following formula: The process is iterated until the system's weighted average data freshness converges.

[0066] VI. Execution Process of Dynamic Energy Redistribution Mechanism During the online deployment phase after training convergence, the following dynamic energy reallocation process is executed after policy inference is completed at each time step: Step 1: Obtain the target access node, flight speed, and transmission power parameters output by the current strategy, and assess the expected energy consumption and remaining energy budget for executing this action.

[0067] Step 2: If the action involves accessing high-risk nodes and the expected additional flight energy consumption is greater than the remaining energy budget, the dynamic energy reallocation mechanism is triggered to divide the safe subset and the critical subset according to the risk probability threshold.

[0068] Step 3: Calculate the hovering power saved when nodes in the safe subset transmit at full power. If the power saving meets the energy consumption gap, update the transmission power parameters of the nodes in the safe subset. Execute the optimal action strategy.

[0069] Step 4: If the power saving cannot meet the energy consumption gap, then the low-weight nodes in the safe subset are removed in ascending order of attention weight, and the node access sequence is replanned until the released energy redundancy meets the flight requirements and return-to-home safety constraints, and the downgraded collection strategy is executed.

[0070] like Figure 3 As shown, simulation results indicate that within the total energy constraint range of 100kJ to 500kJ, the system weighted average data freshness of the proposed method (PPO) is consistently significantly lower than that of the other four comparative algorithms, demonstrating the best data freshness optimization capability, energy utilization efficiency, and scenario adaptability.

[0071] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0072] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in sequences other than those illustrated or described herein.

[0073] The present invention has been further described above with reference to specific embodiments. However, it should be understood that the specific description herein should not be construed as limiting the nature and scope of the present invention. Various modifications made to the above embodiments by those skilled in the art after reading this specification are all within the scope of protection of the present invention.

Claims

1. A method for dynamic data acquisition from unmanned aerial vehicles (UAVs) that incorporates data freshness optimization, characterized in that, The method includes: S1. Dynamically acquire the sensor node set and UAV status; acquire the multi-source heterogeneous environment characteristics of each sensor node, and construct a directed graph topology based on the multi-source heterogeneous environment characteristics, and output the adjacency matrix; S2. Input the adjacency matrix and the multi-source heterogeneous environment features into the graph neural network, output the disaster risk probability of each sensor node, and map the node priority according to the disaster risk probability to output the dynamic urgency weight of each sensor node. S3. Construct an energy constraint model and an optimization model for node states; the energy constraint model includes flight energy consumption and hovering energy consumption; the node state includes data freshness, which is an amount that increases over time and is updated when data is uploaded, and a weighted average data freshness is calculated based on the dynamic urgency weight. S4. Construct a decision-making process model, and perform feature fusion on the state of the UAV and the state of the node through a cross-attention mechanism to output a state vector with fixed dimensions. S5. Solve the decision process model to construct a strategy model for outputting hybrid action vectors; S6. Redistribute energy and execute strategies based on the hybrid action vector.

2. The method for dynamic data acquisition of unmanned aerial vehicles (UAVs) based on data freshness optimization as described in claim 1, characterized in that, The method in step S1 includes: S11. Let the sensor node set be... , of which Location coordinates of each sensor node For a three-dimensional fixed vector; in the first... At a time step, when received When a sudden alarm node requests access, the total number of nodes in the current sensor node set is updated. The updated sensor node set is: ; S12, using drones in the The drone's position is updated based on the flight direction angle and flight speed at each time step; S13. Establish a free-space line-of-sight communication link model between the UAV and the sensor node; calculate the communication link between the UAV and the sensor node when the UAV is hovering. The three-dimensional spatial distance between sensor nodes; S14. Calculate the communication signal gain based on the channel gain per unit distance; and calculate the UAV's effect on the first [unclear - possibly a specific data point or parameter] based on the communication bandwidth, sensor transmission power, Gaussian white noise power, and the aforementioned communication signal gain. The data transmission rate when each sensor node collects data.

3. The method for dynamic data acquisition of unmanned aerial vehicles (UAVs) based on data freshness optimization as described in claim 1, characterized in that, The method in step S1 includes: S15. The multi-source heterogeneous environmental features include static geographical prior features and real-time meteorological dynamic features. The static geographical prior features and real-time meteorological dynamic features are concatenated into an environmental perception feature vector of the sensor node. S16. Construct a dynamic directed graph topology based on the environmental perception feature vectors. ;in, Represents a set of nodes, corresponding to the first... A set of sensor nodes at each time step ; This represents a set of edges, corresponding to the directed connections between nodes; Representing the adjacency matrix; calculating the elements of the adjacency matrix based on the physical laws of thermal convection. The adjacency matrix elements are nodes. To the node The weight of the directed edge.

4. The method for dynamic data acquisition of unmanned aerial vehicles (UAVs) based on data freshness optimization as described in claim 1, characterized in that, The graph neural network includes a spatial feature aggregation layer and a temporal feature evolution layer connected in series. The method in step S2 includes: S21. The environmental perception feature vectors of all sensor nodes are summarized into a full node environmental perception feature matrix. The adjacency matrix and the full node environmental perception feature matrix are weighted and aggregated through graph convolution operation to output the spatial latent feature matrix. S22. In the temporal feature evolution layer, the continuous The spatial latent feature matrix of each time step is input into a gated recurrent unit; the gated recurrent unit includes a reset gate and an update gate; the reset gate is calculated based on the temporal feature vector of the previous time step and the spatial feature vector of the current time step, and the update gate is calculated based on the temporal feature vector of the previous time step and the spatial feature vector of the current time step; the temporal feature vector of the previous time step is modulated based on the reset gate, and a candidate temporal feature vector is calculated by combining it with the spatial feature vector of the current time step; the temporal feature vector of the previous time step and the candidate temporal feature vector are weighted and combined based on the update gate to output the temporal latent feature vector of the current time step; S23. Input the temporal latent feature vector into the multilayer perceptron and output the disaster risk probability of the sensor nodes within the target time window; S24. Based on the disaster risk probability, the static priority weights of each node are reconstructed into dynamic urgency weights through an exponential mapping function; the static priority weights are determined during the system initialization phase.

5. The method for dynamic data acquisition of unmanned aerial vehicles (UAVs) based on data freshness optimization as described in claim 1, characterized in that, The method in step S3 includes: S31. Calculate the drone's propulsion power based on the drone's status and fuselage parameters; S32. Calculate flight energy consumption based on the flight time of the nth time step, calculate the hovering data collection time of the UAV based on the amount of data sent by the sensor node and the transmission rate, and calculate hovering energy consumption based on the hovering data collection time. S33. Construct a total available energy constraint for the system, limiting the sum of the cumulative flight energy consumption and hovering energy consumption of the UAV during the mission cycle to not exceed the preset energy limit; S34. Define the data freshness of each sensor node. When the data upload of the sensor node is completed, update the corresponding data freshness to the initial value. When the data is not uploaded, increase the data freshness over time and do not exceed the preset upper limit. S35. Calculate the weighted average data freshness of the system based on the data freshness of each sensor node and the corresponding dynamic urgency weight.

6. The method for dynamic data acquisition of unmanned aerial vehicles (UAVs) based on data freshness optimization as described in claim 1, characterized in that, The decision process model is a Markov decision process model, and the method in step S4 includes: S41. Convert the UAV state into an ontology state vector. The node state also includes dynamic urgency weight, disaster risk probability, and node maximum transmission power. Convert the node state into a node state vector. Map the ontology state vector to a query vector. Map each node state vector to a key vector and a value vector. S42. Calculate the attention score of the UAV's current state to the sensor node through dot product attention, and obtain the attention weight after normalizing the attention score; S43. Use attention weights to perform a weighted summation of the value vectors of all nodes, and output a global environment feature with constant dimension. S44. The ontology state vector is concatenated with the global environment features to form a state vector with fixed dimensions.

7. The method for dynamic data acquisition of unmanned aerial vehicles (UAVs) based on data freshness optimization as described in claim 6, characterized in that, Step S4 further includes: S45. Construct a hybrid action space, which includes discrete actions for representing node access selection and continuous actions for representing flight speed and transmission power; wherein, the action at each time step is a combination of node access actions, flight speed control parameters, and transmission power control parameters. S46. Define a state transition function to represent the probability distribution of transitioning from the current state to the next state after performing the action; S47. Define a reward function, which includes a term based on weighted average data freshness, a penalty term for data freshness exceeding a preset upper limit, a penalty term for energy consumption, and a reward term for high-risk nodes completing data upload.

8. The method for dynamic data acquisition of unmanned aerial vehicles (UAVs) based on data freshness optimization as described in claim 1, characterized in that, Step S5 includes: S51. The policy model constructed by the proximal policy optimization algorithm is modeled as a probability distribution of choosing an action under a given state. S52. Output discrete and continuous policies respectively through a dual-policy network; the discrete policy outputs the specific probability distribution of discrete actions, and the continuous policy outputs the Gaussian distribution of continuous actions by updating the action mean and action standard deviation. S53. Define the policy-based expected reward as the reward function, and update the dual-policy network using the policy gradient method; S54. Update the weights of the dual-policy network using the gradient ascent method.

9. The method for dynamic data acquisition of unmanned aerial vehicles (UAVs) based on data freshness optimization as described in claim 8, characterized in that, Step S5 further includes: S55. Calculate the ratio of the action probability of the old and new strategies for the discrete strategy and the continuous strategy respectively; S56. Introduce a value network to estimate state value and calculate temporal difference error; calculate temporal difference error based on the reward of the current time step and the state value of the next time step. S57. Based on the time-series difference error, the errors of multiple time steps are weighted and summed according to a preset discount factor to obtain an estimate used to characterize the value of the action. S58. Set the shearing range parameters, and construct a policy objective function containing the shearing function based on the action probability ratio and the estimate; S59. Update the network parameters of the discrete policy and the continuous policy respectively using the gradient ascent method; S510. Construct a value network loss function to characterize the difference between the predicted state value and the target reward, and update the value network parameters based on the value network loss function; introduce a policy entropy function, and construct a total loss function together with the policy objective function and the value network loss function to update the network parameters.

10. The method for dynamic data acquisition of unmanned aerial vehicles (UAVs) based on data freshness optimization as described in claim 7, characterized in that, Step S6 includes: S61. Before executing the hybrid action space, if the current state includes the high-risk node and the disaster risk probability of the sensor node exceeds a preset threshold, then calculate the dynamic risk redistribution of the sensor node. S62. Based on the disaster risk probability and the preset threshold, the remaining sensor nodes to be accessed are divided into a safe subset and a critical subset; S63. Send a transmission power adjustment command to all sensor nodes in the safety subset to set the transmission power to the maximum hardware transmission power and reduce the hovering energy consumption. S64. When the reduced hovering energy consumption is greater than or equal to the flight energy consumption required to perform the action of accessing the target sensor node, the saved hovering power is allocated to the flight energy consumption to fly to the target high-risk node, and the optimal action strategy output in step S5 is executed; when the reduced hovering energy consumption is less than the flight energy consumption required to perform the action of accessing the target sensor node, the sensor nodes of the safe subset with the lowest score are eliminated in ascending order according to the attention weight calculated in step S4, and the reduced hovering energy consumption is greater than or equal to the flight energy consumption required to perform the action of accessing the target sensor node.

Images