Unmanned aerial vehicle data acquisition method based on sensor data freshness

Abstract

The invention discloses an unmanned aerial vehicle data acquisition method based on sensor data freshness, and belongs to the technical field of communication. The invention provides an unmanned aerial vehicle data acquisition method based on sensor data freshness, and aims to solve the problem that in a large-scale area non-uniformly distributed sensor data acquisition scene, an unmanned aerial vehicle cruise acquisition strategy may be difficult to ensure the balance and fairness of data acquisition under the constraint of energy consumption, so that the data freshness is deteriorated. The method adopts a dual-delay depth deterministic strategy gradient algorithm to establish a deep reinforcement learning model, dynamically plans the cruise trajectory of the unmanned aerial vehicle according to the position distribution of ground sensors, dynamically adjusts a data acquisition strategy according to the real-time distribution of sensor data freshness, and determines the energy consumption of the unmanned aerial vehicle by setting the energy consumption constraint of the unmanned aerial vehicle. Dynamic balance and optimization are carried out on the target sensor object and the data acquisition quantity in the cruise acquisition process, so that the data freshness of the system is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of communication technology, and in particular relates to a method for collecting unmanned aerial vehicle data based on the freshness of sensor data. Background Art

[0002] With the continuous development of low-altitude UAV technology, using a single UAV to collect ground sensor data has become an effective means of replacing a large number of manual operations with a small number of machines. However, when collecting data from non-uniformly distributed ground sensors in large-scale areas, the UAV cruise collection strategy may be difficult to ensure the balance and fairness of data collection under energy consumption constraints, resulting in the deterioration of the freshness of some sensor data. Among them, data freshness refers to the average delay from data generation to data collection, which represents the validity and reliability of system data.

[0003] In view of the above problems, the present invention proposes a UAV data collection method based on sensor data freshness. The method uses a double-delay deep deterministic policy gradient algorithm [1] to establish a deep reinforcement learning model. By setting up UAV energy consumption constraints, the target sensor objects and data collection volume in the cruise collection process are dynamically balanced and optimized, thereby effectively improving the data freshness of the system.

[0004] [1]F.Scott,VHHerke,M.David.Addressing Function Approximation Errorin Actor-Critic Methods[C].Proceedings of the 35th International Conference on Machine Learning,Stockholm,Sweden,2018:2587-2601. Summary of the invention

[0005] The present invention aims to solve the above problems of the prior art. A method for collecting drone data based on the freshness of sensor data is proposed. The technical solution of the present invention is as follows:

[0006] A method for collecting drone data based on sensor data freshness comprises the following steps:

[0007] 101. A double-delay deep deterministic policy gradient algorithm is used to construct a deep reinforcement learning model, including a drone action strategy generator, a drone state-action value generator, a drone data collection strategy generator, an experience sample storage area, and a random sample set. The drone's cruise collection time is divided into several equal-length time slots of length s, and the system state remains unchanged in any time slot k. The system state vector [s k ,ak ,r k ,s k+1 ], where s k is the drone state vector at time slot k, a k is the UAV action vector of time slot k, r k is the system reward for time slot k, s k+1 is the drone state vector of time slot k+1, initialize k=0;

[0008] 102. Let k = k + 1, the UAV data collection strategy generator is based on the state vector s k , update the k-time slot system data freshness set in, is the data freshness of sensor i in time slot k, where data freshness represents the average delay from data generation to data collection;

[0009] 103. Obtain s through the drone action strategy generator k The drone motion vector a in the state k , according to the set F k , get system reward r k , according to s k with a k Calculate the k+1 time slot drone state vector s k+1 , the system state vector [s k ,a k ,r k ,s k+1 ] Added experience sample storage area;

[0010] 104. Randomly sample a sample set from the experience sample storage area, and import the sample set into the drone action strategy generator and the drone state-action value generator for training;

[0011] 105. If the energy consumption E of the drone at the end of time slot k reaches the energy consumption threshold E max , jump to step 106, otherwise, jump to step 102;

[0012] 106. The collection work is completed.

[0013] Furthermore, in step 101, the system state vector [s k ,a k ,r k ,s k+1 ], where the UAV state vector s in time slot k k , UAV motion vector a k 、System Rewards k 、UAVs in s k Execute a in the state kAfter the action, the state vector s of the drone entering the k+1 time slot is k+1 , as shown in formulas (1), (2), (3), and (4) respectively:

[0014]

[0015] a k =[(θ k ,φ k ,v k )] (2)

[0016] r k =-α·μ k -β·σ k (3)

[0017]

[0018] In formula (1), (x, y, z) k represents the coordinate vector of the k-time slot UAV, Indicates whether the k-time slot sensor i is within the coverage of the drone. If so, otherwise I represents the set of all sensors in the system; in formula (2), θ k ∈[0,2π] represents the horizontal flight angle of the UAV in the k-time slot, φ k ∈[0,π] represents the vertical flight angle of the UAV in time slot k, v k represents the flight speed of the UAV in the k-time slot; in formula (3), α and β represent weight factors, μ k represents the mean value of the freshness of sensor data in time slot k, σ k represents the standard deviation of the freshness of sensor data in time slot k, where μ k , σ k The calculation method is shown in formulas (5) and (6):

[0019]

[0020]

[0021] In formula (5), f k i represents the data freshness of sensor i at time slot k.

[0022] Furthermore, in step 102, the k-time slot system data freshness set F is updated by the drone data collection strategy generator. k The method is as follows:

[0023] 1) According to the state vector s k, obtain all sensors within the current drone coverage area and add them to the temporary set I′, initialize the data collection time t of the drone in the k time slot k =0;

[0024] 2) According to the data freshness of each sensor i in I′ Arrange the elements in I′ in descending order, take out the first sensor i in the set I′, collect the first data packet on i, and let t k =t k +Δt, where Δt is the acquisition time of a data packet, updates the data freshness of sensor i in time slot k

[0025] 3) If t k <τ, jump to step 2), otherwise jump to step 4);

[0026] 4) According to the data freshness of each sensor i in I′ Update system data freshness set F k ;

[0027] 5) The step ends.

[0028] Furthermore, in step 103, the k-time slot system reward r is obtained. k and k+1 time slot drone state vector s k+1 The method is as follows:

[0029] 6) Initialize system reward r k =0, binary variable γ = 0;

[0030] 7) According to the k-time slot drone state vector s k , UAV motion vector a k , get the coordinate vector (x,y,z) of the drone in time slot k+1 k+1 and the drone state vector s k+1 ;

[0031] 8) According to the coordinate vector (x, y, z) of the drone in the k+1 time slot k+1 Determine whether the drone exceeds the boundary of the data collection area. If so, jump to step 9); otherwise, jump to step 10);

[0032] 9) Let the state vector of the drone at time slot k+1 be s k+1 =s k , binary variable γ = 1;

[0033] 10) According to the data freshness set F of k time slot sensors k , get system reward r k , let r k =rk -γ·p, where p is the penalty value for the drone exceeding the boundary of the data collection area;

[0034] 11) Return the k-time slot system reward r k and the UAV state vector s at time slot k+1 k+1 ;

[0035] 12) The steps are completed.

[0036] Furthermore, the energy consumption E of the drone obtained in step 105 is specifically shown in formula (7):

[0037]

[0038] In formula (7), c1 and c2 represent weight factors related to the weight, wing area, and air density of the UAV, τ represents the time slot length, and v k represents the speed of the UAV in time slot k.

[0039] Furthermore, in step 4), the sensor data freshness set F is updated k When , the method for calculating the freshness of a single sensor is as shown in formula (8):

[0040]

[0041] In formula (8), represents the set of data packets collected by the drone from sensor i up to time slot k, D represents the size of the data packet generated by the sensor, represents the generation time of the jth data packet of sensor i, t i,j It represents the receiving time of the jth data packet from sensor i at the base.

[0042] Furthermore, in step 7), the coordinate vector (x, y, z) of the drone in the k+1 time slot is obtained. k+1 The method is shown in formula (9):

[0043]

[0044] In formula (9), τ represents the time slot length, θ k ∈[0,2π] represents the horizontal flight angle of the UAV in the k-time slot, φ k ∈[0,π] represents the vertical flight angle of the UAV in time slot k, v k represents the flight speed of the UAV in time slot k.

[0045] The advantages and beneficial effects of the present invention are as follows:

[0046] The present invention discloses a method for collecting data from a drone based on the freshness of sensor data. In a sensor data collection scenario with non-uniform distribution in a large-scale area, the drone cruise collection strategy may be difficult to ensure the balance and fairness of data collection under energy consumption constraints, which in turn leads to the problem of deterioration of data freshness. In view of the above problems, the present invention proposes a method for collecting data from a drone based on the freshness of sensor data. The method uses a double-delay deep deterministic policy gradient algorithm to establish a deep reinforcement learning model, dynamically plans the cruise trajectory of the drone according to the location distribution of ground sensors, and dynamically adjusts the data collection strategy according to the real-time distribution of sensor data freshness. By setting up drone energy consumption constraints, the target sensor objects and data collection volume in the cruise collection process are dynamically balanced and optimized, thereby effectively improving the data freshness of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 The present invention provides a preferred embodiment of a flow chart of a drone data collection method based on sensor data freshness. DETAILED DESCRIPTION

[0048] The following will describe the technical solutions in the embodiments of the present invention in detail in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention.

[0049] The technical solution of the present invention to solve the above technical problems is:

[0050] The concepts and models involved in the present invention are as follows:

[0051] The present invention assumes that the area where the drone collects data is within the effective communication range of the base. When the drone collects the data packet of the sensor, it can transmit the data to the base through line-of-sight communication. The drone-assisted IoT data collection system model considered in the present invention consists of a drone, a data base and a large number of ground sensors. Assume that the sensor set I = {i} is in L×L m 2 The area is randomly distributed. During the UAV's data collection mission, the position of the sensor node is fixed, and the sensor generates fixed-size data packets at the same rate. Initially, the number of data packets of different sensors is different. In addition, the present invention assumes that the base has a central control unit responsible for collecting system information in the network. Based on this information, the central control unit will issue instructions to guide the movement and operation of the UAV.

[0052] The technical solution of the present invention is described as follows:

[0053] 1. System state vector [s k ,a k ,r k ,s k+1 ]

[0054] The state vector s of the UAV in time slot k k , UAV motion vector a k 、System Rewards k 、UAVs in s k Execute a in the state k After the action, the state vector s of the drone entering the k+1 time slot is k+1 , as shown in formulas (1), (2), (3), and (4) respectively:

[0055]

[0056] a k =[(θ k ,φ k ,v k )] (2)

[0057] r k =-α·μ k -β·σ k (3)

[0058]

[0059] In formula (1), (x, y, z) k represents the coordinate vector of the k-time slot UAV, Indicates whether the k-time slot sensor i is within the coverage of the drone. If so, otherwise I represents the set of all sensors in the system; in formula (2), θ k ∈[0,2π] represents the horizontal flight angle of the UAV in the k-time slot, φ k ∈[0,π] represents the vertical flight angle of the UAV in time slot k, v k represents the flight speed of the UAV in the k-time slot; in formula (3), α and β represent weight factors, μ k represents the mean value of the freshness of sensor data in time slot k, σ k represents the standard deviation of the freshness of sensor data in time slot k, where μ k , σ k The calculation method is shown in formulas (5) and (6):

[0060]

[0061] In formula (5), f k i represents the data freshness of sensor i in time slot k;

[0062] 2. Calculation method of UAV energy consumption E

[0063] The energy consumption E method of UAV is shown in formula (7):

[0064]

[0065] In formula (7), c1 and c2 represent weight factors related to the weight of the drone, wing area, air density, etc., τ represents the time slot length, and v k represents the speed of the UAV in time slot k;

[0066] 3. Sensor data freshness f k i Calculation method

[0067] Update the sensor data freshness set F k When , the method for calculating the freshness of a single sensor is as shown in formula (8):

[0068]

[0069] In formula (8), represents the set of data packets collected by the drone from sensor i up to time slot k, D represents the size of the data packet generated by the sensor, represents the generation time of the jth data packet of sensor i, t i,j represents the reception time of the jth data packet from sensor i at the base;

[0070] 4. Get the coordinate vector (x, y, z) of the drone in the k+1 time slot k+1 Methods

[0071] Get the drone coordinate vector (x, y, z) of the drone in the k+1 time slot k+1 The method is shown in formula (9):

[0072]

[0073] In formula (9), τ represents the time slot length, θ k ∈[0,2π] represents the horizontal flight angle of the UAV in the k-time slot, φ k ∈[0,π] represents the vertical flight angle of the UAV in time slot k, v k represents the flight speed of the UAV in time slot k;

[0074] 5. Sub-algorithm 1: Update the k-time slot system data freshness set F k

[0075] Step 1: According to the state vector s k , obtain all sensors within the current drone coverage area and add them to the temporary set I′, initialize the data collection time t of the drone in the k time slotk =0;

[0076] Step 2: According to the data freshness of each sensor i in I′ Arrange the elements in I′ in descending order, take out the first sensor i in the set I′, collect the first data packet on i, and let t k =t k +Δt, where Δt is the acquisition time of a data packet, updates the data freshness of sensor i in time slot k

[0077] Step 3: If t k <τ, jump to step 2, otherwise jump to step 4;

[0078] Step 4: According to the data freshness of each sensor i in I′ Update system data freshness set F k ;

[0079] Step 5: End of step.

[0080] 6. Sub-algorithm 2: Obtaining the k-time slot system reward r k and k+1 time slot drone state vector s k+1

[0081] Step 1: Initialize system reward r k =0, binary variable γ = 0;

[0082] Step 2: According to the k-time slot drone state vector s k , UAV motion vector a k , get the coordinate vector (x,y,z) of the drone in time slot k+1 k+1 and the drone state vector s k+1 ;

[0083] Step 3: According to the coordinate vector (x, y, z) of the drone at time slot k+1 k+1 Determine whether the drone exceeds the boundary of the data collection area. If so, jump to step 4; otherwise, jump to step 5.

[0084] Step 4: Let the drone state vector s at time slot k+1 be k+1 =s k , binary variable γ = 1;

[0085] Step 5: According to the data freshness set F of the k-time slot sensor k , get system reward r k , let r k =r k -γ·p, where p is the penalty value for the drone exceeding the boundary of the data collection area;

[0086] Step 6: Return the k-slot system reward r k and the UAV state vector s at time slot k+1 k+1 ;

[0087] Step 7: End of the step.

[0088] A method for collecting drone data based on sensor data freshness, the specific implementation method of which includes the following steps:

[0089] Step 1: Use the double-delay deep deterministic policy gradient algorithm to build a deep reinforcement learning model, including the drone action strategy generator, drone state-action value generator, drone data collection strategy generator, experience sample storage area, and random sample set. The drone's cruise collection time is divided into several equal-length time slots of length τ, and the system state remains unchanged in any time slot k. Establish the system state vector [s k ,a k ,r k ,s k+1 ], where s k is the drone state vector of time slot k, a k is the UAV motion vector of time slot k, r k is the system reward for time slot k, s k+1 is the drone state vector of time slot k+1, initialize k=0;

[0090] Step 2: Let k = k + 1, the UAV data collection strategy generator is based on the state vector s k , call sub-algorithm 1 to update the k-time slot system data freshness set in, is the data freshness of sensor i in time slot k, where data freshness represents the average delay from data generation to data collection;

[0091] Step 3: According to the set F k , call sub-algorithm 2 to obtain the system reward r k , obtained through the drone action strategy generator k The drone motion vector a in the state k , according to s k with a k Calculate the k+1 time slot drone state vector s k+1 , the system state vector [s k ,a k ,r k ,s k+1 ] Added experience sample storage area;

[0092] Step 4: Randomly sample a sample set from the experience sample storage area, and import the sample set into the drone action strategy generator and the drone state-action value generator for training;

[0093] Step 5: If the energy consumption E of the drone at the end of time slot k reaches the energy consumption threshold E max , jump to step 106, otherwise, jump to step 102;

[0094] Step 6: The collection work is completed.

[0095] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions.

[0096] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0097] The above embodiments should be understood to be only used to illustrate the present invention and not to limit the protection scope of the present invention. After reading the contents of the present invention, technicians can make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A method for collecting drone data based on sensor data freshness, characterized in that: The following steps are involved:

101. A double-delay deep deterministic policy gradient algorithm is used to construct a deep reinforcement learning model, including a drone action strategy generator, a drone state-action value generator, a drone data collection strategy generator, an experience sample storage area, and a random sample set. The drone's cruise collection time is divided into several equal-length time slots of length τ, and the system state remains unchanged in any time slot k. The system state vector [s k ,a k ,r k ,s k+1 ], where s k is the drone state vector at time slot k, a k is the UAV action vector of time slot k, r k is the system reward for time slot k, s k+1 is the drone state vector of time slot k+1, initialize k=0; 102. Let k = k + 1, the UAV data collection strategy generator is based on the state vector s k , update the k-time slot system data freshness set in, is the data freshness of sensor i in time slot k, where data freshness represents the average delay from data generation to data collection; 103. Obtain s through the drone action strategy generator k The drone motion vector a in the state k , according to the set F k , get system reward r k , according to s k with a k Calculate the k+1 time slot drone state vector s k+1 , the system state vector [s k ,a k ,r k ,s k+1 ] Added experience sample storage area; 104. Randomly sample a sample set from the experience sample storage area, and import the sample set into the drone action strategy generator and the drone state-action value generator for training; 105. If the energy consumption E of the drone at the end of time slot k reaches the energy consumption threshold E max , jump to step 106, otherwise, jump to step 102; 106. The collection work is completed.

2. The method for collecting drone data based on sensor data freshness according to claim 1, characterized in that: In step 101, a system state vector [s k ,a k ,r k ,s k+1 ], where the UAV state vector s in time slot k k , UAV motion vector a k 、System Rewards k 、UAVs in s k Execute a in the state k After the action, the state vector s of the drone entering the k+1 time slot is k+1 , as shown in formulas (1), (2), (3), and (4) respectively: a k =[(θ k ,f k ,v k )] (2) r k =-a·m k -b·s k (3) In formula (1), (x, y, z) k represents the coordinate vector of the k-time slot UAV, Indicates whether the k-time slot sensor i is within the coverage of the drone. If so, otherwise I represents the set of all sensors in the system; in formula (2), θ k ∈[0,2π] represents the horizontal flight angle of the UAV in the k-time slot, φ k ∈[0,π] represents the vertical flight angle of the UAV in time slot k, v k represents the flight speed of the UAV in the k-time slot; in formula (3), α and β represent weight factors, μ k represents the mean value of the freshness of sensor data in time slot k, σ k represents the standard deviation of the freshness of sensor data in time slot k, where μ k , σ k The calculation method is shown in formulas (5) and (6): In formula (5), f k i represents the data freshness of sensor i at time slot k.

3. The method for collecting drone data based on sensor data freshness according to claim 1, characterized in that: In step 102, the k-time slot system data freshness set F is updated by the drone data collection strategy generator. k The method is as follows: 1) According to the state vector s k , obtain all sensors within the current drone coverage area and add them to the temporary set I′, initialize the data collection time t of the drone in the k time slot k =0; 2) According to the data freshness of each sensor i in I′ Arrange the elements in I′ in descending order, take out the first sensor i in the set I′, collect the first data packet on i, and let t k =t k +Δt, where Δt is the acquisition time of a data packet, updates the data freshness of sensor i in time slot k 3) If t k <τ, jump to step 2), otherwise jump to step 4); 4) According to the data freshness of each sensor i in I′ Update system data freshness set F k ; 5) The step ends.

4. The method for collecting drone data based on sensor data freshness according to claim 1, characterized in that: In step 103, the k-time slot system reward r is obtained. k and k+1 time slot drone state vector s k+1 The method is as follows: 6) Initialize system reward r k =0, binary variable γ=0; 7) According to the k-time slot drone state vector s k , UAV motion vector a k , get the coordinate vector (x,y,z) of the drone in time slot k+1 k+1 and the drone state vector s k+1 ; 8) According to the coordinate vector (x, y, z) of the drone in the k+1 time slot k+1 Determine whether the drone exceeds the boundary of the data collection area. If so, jump to step 9); otherwise, jump to step 10); 9) Let the state vector of the drone at time slot k+1 be s k+1 =s k , binary variable γ = 1; 10) According to the data freshness set F of k time slot sensors k , get system reward r k , let r k =r k -γ·p, where p is the penalty value for the drone exceeding the boundary of the data collection area; 11) Return the k-time slot system reward r k and the UAV state vector s at time slot k+1 k+1 ; 12) The steps are completed.

5. The method for collecting drone data based on sensor data freshness according to claim 1, characterized in that: The energy consumption E of the drone obtained in step 105 is specifically shown in formula (7): In formula (7), c1 and c2 represent weight factors related to the weight, wing area, and air density of the UAV, τ represents the time slot length, and v k represents the speed of the UAV in time slot k.

6. The method for collecting drone data based on sensor data freshness according to claim 3, characterized in that: In step 4), the sensor data freshness set F is updated. k When , the method for calculating the freshness of a single sensor is as shown in formula (8): In formula (8), represents the set of data packets collected by the drone from sensor i up to time slot k, D represents the size of the data packet generated by the sensor, represents the generation time of the jth data packet of sensor i, t i,j It represents the receiving time of the jth data packet from sensor i at the base.

7. The method for collecting drone data based on sensor data freshness according to claim 4, characterized in that: In step 7), the coordinate vector (x, y, z) of the drone in the k+1 time slot is obtained. k+1 The method is shown in formula (9): In formula (9), τ represents the time slot length, θ k ∈[0,2π] represents the horizontal flight angle of the UAV in the k-time slot, φ k ∈[0,π] represents the vertical flight angle of the UAV in time slot k, v k represents the flight speed of the UAV in time slot k.

Images