An unmanned aerial vehicle data acquisition method based on sensor data freshness

Abstract

The application claims a kind of unmanned aerial vehicle data acquisition method based on sensor data freshness, belongs to communication technical field.For the non-uniform distribution of sensor data acquisition scene in large-scale area, the data acquisition balance and fairness of unmanned aerial vehicle cruise collection strategy may be difficult to guarantee under energy consumption constraint, thereby leading to the problem of data freshness deterioration, a kind of unmanned aerial vehicle data acquisition method based on sensor data freshness is proposed.The method adopts double delay deep deterministic policy gradient algorithm to establish deep reinforcement learning model, according to the position distribution of ground sensor, dynamically plans the cruise trajectory of unmanned aerial vehicle, according to the real-time distribution of sensor data freshness, dynamically adjusts data collection strategy, through setting up the energy consumption constraint of unmanned aerial vehicle, dynamically balances and optimizes the target sensor object and data collection amount in cruise collection process, thereby effectively improving the data freshness of system.

Description

Technical Field

[0001] This invention belongs to the field of communication technology, and specifically relates to a method for collecting UAV data based on the freshness of sensor data. Background Technology

[0002] With the continuous development of low-altitude unmanned aerial vehicle (UAV) technology, using a single UAV to collect ground sensor data has become an effective means of replacing a large amount of manual labor with a small number of machines. However, when collecting data from ground sensors that are not uniformly distributed over a large area, the UAV cruise acquisition strategy may struggle to guarantee the balance and fairness of data collection under energy consumption constraints, leading to the deterioration of the freshness of some sensor data. Data freshness refers to the average time delay from data generation to acquisition, representing the validity and reliability of the system data.

[0003] To address the above problems, this invention proposes a UAV data acquisition method based on sensor data freshness. This method uses a dual-delay deep deterministic strategy gradient algorithm [1] to establish a deep reinforcement learning model. By setting UAV energy consumption constraints, the target sensor objects and data acquisition volume during the cruise acquisition process are dynamically balanced and optimized, thereby effectively improving the data freshness of the system.

[0004] [1]F.Scott,VHHerke,M.David.Addressing Function Approximation Errorin Actor-Critic Methods[C].Proceedings of the 35th International Conference on Machine Learning,Stockholm,Sweden,2018:2587-2601. Summary of the Invention

[0005] This invention aims to solve the problems of the prior art mentioned above. It proposes a UAV data acquisition method based on sensor data freshness. The technical solution of this invention is as follows:

[0006] A method for drone data acquisition based on sensor data freshness includes the following steps:

[0007] 101. A deep reinforcement learning model is constructed using a dual-delay deep deterministic policy gradient algorithm, including a UAV action policy generator, a UAV state-action value generator, a UAV data acquisition policy generator, an experience sample storage area, and a random sample set. The UAV's cruise acquisition time is divided into several equal-length time slots of length s, and the system state remains unchanged within any time slot k. A system state vector [s] is established. k ,ak ,r k ,s k+1 ], where s k Let a be the UAV state vector for time slot k. k Let r be the UAV motion vector for time slot k. k For the system reward of slot k, s k+1 Let k+1 be the UAV state vector, initialized to k=0;

[0008] 102. Let k = k + 1, the UAV data acquisition strategy generator, based on the state vector s k Update the data freshness set of the k-slot system. in, Let i be the data freshness of time slot k sensor i, where data freshness represents the average time delay from data generation to acquisition;

[0009] 103. Obtain s through the drone motion strategy generator k UAV motion vector a in state k According to set F k Receive system reward r k According to s k With a k Calculate and obtain the UAV state vector s in time slot k+1 k+1 , the system state vector [s k ,a k ,r k ,s k+1 Add to the experience sample storage area;

[0010] 104. Randomly sample the sample set from the experience sample storage area, and import the sample set into the UAV action policy generator and the UAV state-action value generator for training respectively;

[0011] 105. If the energy consumption E of the drone up to time slot k reaches the energy consumption threshold E max If yes, proceed to step 106; otherwise, proceed to step 102.

[0012] 106. Data collection is complete.

[0013] Furthermore, in step 101, a system state vector [s] is established. k ,a k ,r k ,s k+1 ], where the UAV state vector s in time slot k k UAV motion vector a k System rewards r k Drones in s k Execute a in state kAfter the action, the UAV state vector s enters time slot k+1. k+1 As shown in formulas (1), (2), (3), and (4) respectively:

[0014]

[0015] a k =[(θ k ,φ k ,v k (2)

[0016] r k =-α·μ k -β·σ k (3)

[0017]

[0018] In formula (1), (x,y,z) k This represents the coordinate vector of the k-slot UAV. This indicates whether time slot i is within the drone's coverage area. If it is, then... otherwise I represents the set of all sensors in the system; in formula (2), θ k ∈[0,2π] represents the horizontal flight angle of the k-slot UAV, φ k ∈[0,π] represents the vertical flight angle of the k-slot UAV, v k The value represents the flight speed of the UAV in time slot k; in formula (3), α and β represent weighting factors, and μ k σ represents the mean of the freshness of k-slot sensor data. k The standard deviation of the freshness of k-slot sensor data is represented by μ. k , σ k The calculation method is shown in formulas (5) and (6):

[0019]

[0020]

[0021] In formula (5), f k i This represents the data freshness of sensor i in time slot k.

[0022] Furthermore, in step 102, the k-slot system data freshness set F is updated through the UAV data acquisition strategy generator. k The method is as follows:

[0023] 1) Based on the state vector s kObtain all sensors within the current drone's coverage area and add them to a temporary set I′. Initialize the drone's data acquisition time t within time slot k. k =0;

[0024] 2) Based on the data freshness of each sensor i in I′ Sort the elements in set I′ in descending order, take the first sensor i from set I′, collect the first data packet on i, and let t k =t k +Δt, where Δt is the acquisition duration of one data packet, updates the data freshness of sensor i in time slot k.

[0025] 3) If t k <τ, jump to step 2); otherwise, jump to step 4.

[0026] 4) Based on the data freshness of each sensor i in I′ Update the system data freshness set F k ;

[0027] 5) End of steps.

[0028] Furthermore, in step 103, the system reward r for the k-slot is obtained. k and the UAV state vector s in time slot k+1 k+1 The method is as follows:

[0029] 6) Initialize system reward r k =0, binary variable γ = 0;

[0030] 7) Based on the k-slot UAV state vector s k UAV motion vector a k Obtain the UAV coordinate vector (x, y, z) for time slot k+1. k+1 and the UAV state vector s k+1 ;

[0031] 8) Based on the UAV coordinate vector (x, y, z) of time slot k+1 k+1 Determine whether the drone has exceeded the boundary of the data collection area. If it has, proceed to step 9; otherwise, proceed to step 10.

[0032] 9) Let the UAV state vector s in time slot k+1 be... k+1 =s k The binary variable γ = 1;

[0033] 10) Based on the data freshness set F of the k-slot sensor k Receive system reward r k , let r k =rk -γ·p, where p is the penalty value for the drone exceeding the boundary of the data acquisition area;

[0034] 11) Return the system reward r for slot k. k The UAV state vector s in time slot k+1 k+1 ;

[0035] 12) Step 12) End.

[0036] Furthermore, the method for obtaining the UAV energy consumption E in step 105 is specifically as shown in formula (7):

[0037]

[0038] In formula (7), c1 and c2 represent weighting factors related to the weight, wing area, and air density of the UAV, τ represents the time slot length, and v k This represents the speed of the UAV in time slot k.

[0039] Furthermore, in step 4), the sensor data freshness set F is updated. k The method for calculating the freshness of a single sensor is shown in formula (8):

[0040]

[0041] In formula (8), This represents the set of data packets collected by the UAV from sensor i up to time slot k, where D represents the size of the data packets generated by the sensor. t represents the generation time of the j-th data packet from sensor i. i,j This indicates the time when the base receives the j-th data packet from sensor i.

[0042] Furthermore, in step 7), the UAV coordinate vector (x, y, z) in time slot k+1 is obtained. k+1 The method is shown in formula (9):

[0043]

[0044] In formula (9), τ represents the time slot length, and θ k ∈[0,2π] represents the horizontal flight angle of the k-slot UAV, φ k ∈[0,π] represents the vertical flight angle of the k-slot UAV, v k This represents the flight speed of the UAV in time slot k.

[0045] The advantages and beneficial effects of this invention are as follows:

[0046] This invention discloses a UAV data acquisition method based on sensor data freshness. In scenarios where sensor data is non-uniformly distributed over large areas, UAV cruise acquisition strategies may struggle to guarantee the balance and fairness of data acquisition under energy consumption constraints, leading to a deterioration in data freshness. To address this issue, this invention proposes a UAV data acquisition method based on sensor data freshness. This method employs a dual-delay deep deterministic strategy gradient algorithm to establish a deep reinforcement learning model. Based on the location distribution of ground sensors, it dynamically plans the UAV's cruise trajectory. Based on the real-time distribution of sensor data freshness, it dynamically adjusts the data acquisition strategy. By establishing UAV energy consumption constraints, it dynamically balances and optimizes the target sensor objects and data acquisition volume during the cruise acquisition process, thereby effectively improving the system's data freshness. Attached Figure Description

[0047] Figure 1 This is a flowchart of a preferred embodiment of the present invention for a UAV data acquisition method based on sensor data freshness. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and thoroughly described below with reference to the accompanying drawings. The described embodiments are merely some embodiments of the present invention.

[0049] The technical solution of the present invention to solve the above-mentioned technical problems is:

[0050] The concepts and models involved in this invention are as follows:

[0051] This invention assumes that the area where the UAV collects data is within the effective communication range of the base station. When the UAV collects data packets from the sensors, it can transmit the data to the base station via line-of-sight communication. The UAV-assisted IoT data acquisition system model considered in this invention consists of a UAV, a data base station, and a large number of ground sensors. It is assumed that the sensor set I = {i} is located in L × L m. 2 The sensor nodes are randomly distributed across the data acquisition area. During the data acquisition mission, the locations of the sensor nodes are fixed, and the sensors generate data packets of a fixed size at the same rate. Initially, the number of data packets generated by different sensors varies. Furthermore, this invention assumes that the base station has a central control unit responsible for collecting system information from the network. Based on this information, the central control unit will issue commands to guide the movement and operation of the drone.

[0052] The technical solution of the present invention is described as follows:

[0053] 1. System state vector [s] k ,a k ,r k ,s k+1 ]

[0054] The UAV state vector s in time slot k k UAV motion vector a k System rewards r k Drones in s k Execute a in state k After the action, the UAV state vector s enters time slot k+1. k+1 As shown in formulas (1), (2), (3), and (4) respectively:

[0055]

[0056] a k =[(θ k ,φ k ,v k (2)

[0057] r k =-α·μ k -β·σ k (3)

[0058]

[0059] In formula (1), (x,y,z) k This represents the coordinate vector of the k-slot UAV. This indicates whether time slot i is within the drone's coverage area. If it is, then... otherwise I represents the set of all sensors in the system; in formula (2), θ k ∈[0,2π] represents the horizontal flight angle of the k-slot UAV, φ k ∈[0,π] represents the vertical flight angle of the k-slot UAV, v k The value represents the flight speed of the UAV in time slot k; in formula (3), α and β represent weighting factors, and μ k σ represents the mean of the freshness of k-slot sensor data. k The standard deviation of the freshness of k-slot sensor data is represented by μ. k , σ k The calculation method is shown in formulas (5) and (6):

[0060]

[0061] In formula (5), f k i This indicates the data freshness of sensor i in time slot k;

[0062] 2. Calculation method for drone energy consumption E

[0063] The method for calculating the energy consumption E of the UAV is shown in formula (7):

[0064]

[0065] In formula (7), c1 and c2 represent weighting factors related to UAV weight, wing area, air density, etc., τ represents time slot length, and v k This represents the velocity of the UAV in time slot k;

[0066] 3. Sensor data freshness f k i Calculation method

[0067] Update the sensor data freshness set F k The method for calculating the freshness of a single sensor is shown in formula (8):

[0068]

[0069] In formula (8), This represents the set of data packets collected by the UAV from sensor i up to time slot k, where D represents the size of the data packets generated by the sensor. t represents the generation time of the j-th data packet from sensor i. i,j This indicates the time when the base receives the j-th data packet from sensor i;

[0070] 4. Obtain the coordinate vector (x, y, z) of the UAV in time slot k+1. k+1 Method

[0071] Obtain the UAV coordinate vector (x, y, z) in time slot k+1. k+1 The method is shown in formula (9):

[0072]

[0073] In formula (9), τ represents the time slot length, and θ k ∈[0,2π] represents the horizontal flight angle of the k-slot UAV, φ k ∈[0,π] represents the vertical flight angle of the k-slot UAV, v k This represents the flight speed of the UAV in time slot k;

[0074] 5. Sub-algorithm 1: Update the data freshness set F of the k-slot system k

[0075] Step 1: Based on the state vector s k Obtain all sensors within the current drone's coverage area and add them to a temporary set I′. Initialize the drone's data acquisition time t within time slot k.k =0;

[0076] Step 2: Based on the data freshness of each sensor i in I′ Sort the elements in set I′ in descending order, take the first sensor i from set I′, collect the first data packet on i, and let t k =t k +Δt, where Δt is the acquisition duration of one data packet, updates the data freshness of sensor i in time slot k.

[0077] Step 3: If t k If <τ, proceed to step 2; otherwise, proceed to step 4.

[0078] Step 4: Based on the data freshness of each sensor i in I′ Update the system data freshness set F k ;

[0079] Step 5: End of steps.

[0080] 6. Sub-algorithm 2: Obtaining the system reward r in k time slots k and the UAV state vector s in time slot k+1 k+1

[0081] Step 1: Initialize system rewards r k =0, binary variable γ = 0;

[0082] Step 2: Based on the k-slot UAV state vector s k UAV motion vector a k Obtain the UAV coordinate vector (x, y, z) for time slot k+1. k+1 and the UAV state vector s k+1 ;

[0083] Step 3: Based on the UAV coordinate vector (x, y, z) of time slot k+1 k+1 Determine if the drone has exceeded the boundary of the data collection area. If it has, proceed to step 4; otherwise, proceed to step 5.

[0084] Step 4: Let the UAV state vector s in time slot k+1 be... k+1 =s k The binary variable γ = 1;

[0085] Step 5: Based on the data freshness set F of the k-slot sensor k Receive system reward r k , let r k =r k -γ·p, where p is the penalty value for the drone exceeding the boundary of the data acquisition area;

[0086] Step 6: Return the system reward r for slot k. k The UAV state vector s in time slot k+1 k+1 ;

[0087] Step 7: End of steps.

[0088] A method for drone data acquisition based on sensor data freshness, the specific implementation of which includes the following steps:

[0089] Step 1: Construct a deep reinforcement learning model using the dual-delay deep deterministic policy gradient algorithm, including a UAV action policy generator, a UAV state-action value generator, a UAV data acquisition policy generator, an experience sample storage area, and a random sample set. Divide the UAV's cruise acquisition time into several equal-length time slots of length τ, and ensure that the system state remains unchanged within any time slot k. Establish the system state vector [s]. k ,a k ,r k ,s k+1 ], where s k Let a be the UAV state vector for time slot k. k The UAV motion vector for time slot k, r k For the system reward of slot k, s k+1 Let k+1 be the UAV state vector, initialized to k=0;

[0090] Step 2: Let k = k + 1, the UAV data acquisition strategy generator generates the data based on the state vector s. k Sub-algorithm 1 is invoked to update the data freshness set of the k-slot system. in, Let i be the data freshness of time slot k sensor i, where data freshness represents the average time delay from data generation to acquisition;

[0091] Step 3: Based on set F k Call sub-algorithm 2 to obtain the system reward r. k s is obtained through the drone action strategy generator k UAV motion vector a in state k According to s k With a k Calculate and obtain the UAV state vector s in time slot k+1 k+1 , the system state vector [s k ,a k ,r k ,s k+1 Add to the experience sample storage area;

[0092] Step 4: Randomly sample the sample set from the experience sample storage area, and import the sample set into the UAV action policy generator and the UAV state-action value generator for training respectively;

[0093] Step 5: If the drone's energy consumption E up to time slot k reaches the energy consumption threshold E max If yes, proceed to step 106; otherwise, proceed to step 102.

[0094] Step 6: Data collection is complete.

[0095] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions.

[0096] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0097] The above embodiments should be understood as illustrative only and not as limiting the scope of protection of the present invention. After reading the description of the present invention, those skilled in the art can make various alterations or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A method for acquiring UAV data based on sensor data freshness, characterized in that, Includes the following steps: A deep reinforcement learning model is constructed using a dual-delay deep deterministic policy gradient algorithm, including a UAV action policy generator, a UAV state-action value generator, a UAV data acquisition policy generator, an experience sample storage area, and a random sample set. The UAV's cruise acquisition time is divided into several intervals of length [missing information]. Equal-length time slots, and arbitrary time slots With the internal system state unchanged, establish the system state vector. ,in, for UAV state vector in time slot, for UAV motion vectors in time slots for System rewards for time slots for The UAV state vector of the time slot is initialized. ; make The UAV data acquisition strategy generator is based on the state vector ,renew Time-slot system data freshness set ,in, for Time slot sensor Data freshness refers to the average time between data generation and data collection. Obtained through the drone action strategy generator UAV motion vector in state According to the set Receive system rewards ,according to and Calculation obtained Time-slot UAV state vector , system state vector Add to the experience sample storage area; A sample set is obtained by randomly sampling from the experience sample storage area, and the sample set is imported into the UAV action policy generator and the UAV state-action value generator for training, respectively. If it ends by Time slot drone energy consumption Reaching the energy consumption threshold Jump to the end, otherwise, jump to ; The data collection is complete; The update via the UAV data acquisition strategy generator Time-slot system data freshness set The method is as follows: A1. Based on the state vector Obtain all sensors within the current drone's coverage area and add them to a temporary set. Initialize the drone Data acquisition time within the time slot ; A2, according to Each sensor Data freshness ,right Sort the elements in descending order and retrieve the set. The first sensor ,collection The first data packet on, let ,in, Update for the duration of a data packet collection. Time slot sensor Data freshness ; A3, if If yes, proceed to step A2; otherwise, proceed to step A4. A4, according to Each sensor Data freshness Update the system data freshness set ; A5. End of steps.

2. The UAV data acquisition method based on sensor data freshness according to claim 1, characterized in that, Establish system state vector ,in, UAV state vector in time slot UAV motion vectors System rewards Drones Execute in state After the action, enter UAV state vector in time slot As shown in formulas (1), (2), (3), and (4) respectively: (1)； (2)； (3)； (4)； In formula (1), express The coordinate vector of a time-slotted UAV. express Time slot sensor Is it within the drone's coverage area? If it is within the drone's coverage area, then... ,otherwise , This represents the set of all sensors in the system; in formula (2), express The horizontal flight angle of the time-slot drone express Vertical flight angle of time-slot drones express The flight speed of the time-slotted UAV; in formula (3), , Indicates the weighting factor. express The mean freshness of time-slot sensor data. express The standard deviation of time-slot sensor data freshness, of which , The calculation method is shown in formulas (5) and (6): (5)； (6)； In formula (5), express Time slot sensor Data freshness.

3. The UAV data acquisition method based on sensor data freshness according to claim 1, characterized in that, The acquisition Time slot system rewards and Time-slot UAV state vector The method is as follows: B1. Initialize System Rewards binary variables ; B2, according to Time-slot UAV state vector UAV motion vectors ,get UAV coordinate vector in time slot and UAV state vector ; B3, according to UAV coordinate vector in time slot Determine whether the drone has exceeded the boundary of the data collection area. If it has, proceed to step B4; otherwise, proceed to step B5. B4, Order UAV state vector in time slot binary variables ; B5. According to Data freshness set of time slot sensors Receive system rewards ,make ,in, The penalty value for the drone exceeding the boundary of the data collection area; B6, Return Time slot system rewards and UAV state vector in time slot ; B7. Step complete.

4. The UAV data acquisition method based on sensor data freshness according to claim 1, characterized in that, The obtained drone energy consumption The method is as shown in formula (7): (7)； In formula (7), and This represents the weighting factors related to the drone's weight, wing area, and air density. Indicates the time slot length. express The speed of time-slot drones.

5. The UAV data acquisition method based on sensor data freshness according to claim 1, characterized in that, Update the sensor data freshness set The method for calculating the freshness of a single sensor is shown in formula (8): (8)； In formula (8), Indicates the deadline Time-slot UAVs collect data from sensors The set of data packets, Indicates the size of the data packets generated by the sensor. Indicates sensor The The generation time of each data packet The base received the sensor. The The time it takes to receive a data packet.

6. The UAV data acquisition method based on sensor data freshness according to claim 3, characterized in that, Obtaining drones in UAV coordinate vector in time slot The method is as shown in formula (9): (9)； In formula (9), Indicates the time slot length. express The horizontal flight angle of the time-slot drone express Vertical flight angle of time-slot drones express The flight speed of the time-slot drone.

Images