A method for collecting data from unmanned aerial vehicles
By modeling the network utility function in the drone data collection system and using a dual-delay depth deterministic strategy gradient network to optimize the UAV trajectory and SN association strategy, the problem of low data collection efficiency in the existing technology is solved, and the network is realized Maximize utility.
Patent Information
- Application Number
- CN202211144132.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-20
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2042-09-20
AI Technical Summary
The prior art is difficult to effectively optimize the impact of SN clustering, SN transmission energy consumption and UAV data collection energy consumption in UAV data collection systems on network utility, resulting in low data collection efficiency.
By modeling the network utility function, combining the opposite number of energy consumption required for SN transmission and UAV to collect data, a dual-delay depth deterministic strategy gradient network is used to optimize the UAV trajectory and SN association strategy to achieve resource scheduling and bandwidth allocation.
With comprehensive consideration of SN bandwidth allocation and resource scheduling limitations, joint optimization of drone trajectory and sensor node association strategy is realized, and network utility is maximized.
Smart Images

Figure CN115550866B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of wireless communications and relates to a method for collecting data from an unmanned aerial vehicle. Background Art
[0002] In recent years, due to the development of UAV manufacturing technology and the reduction of costs, UAVs have been widely used in civil and commercial fields. In some IoT data collection applications, wireless sensor nodes need to collect environmental data and transmit them to the aggregation node. Due to the limited energy of SN and the long distance between SN and the aggregation node, it is difficult to complete data collection efficiently. The use of UAV for data collection has received more and more attention. Compared with traditional ground communication systems, the optimization of UAV trajectory and association strategy can effectively improve the efficiency of data collection. Related studies have shown that the trajectory and association strategy of UAV will have a significant impact on system performance.
[0003] At present, there are some literatures that study the UAV data collection problem. For example, a UAV data collection strategy based on minimizing the time age is proposed. However, few existing studies jointly consider the impact of SN clustering, SN sending energy consumption and the energy consumption required for UAV data collection on network utility, making it difficult to achieve efficient data collection performance optimization. Summary of the invention
[0004] In view of this, the purpose of the present invention is to provide a drone data collection method, for a UAV-assisted IoT data collection system comprising a UAV and multiple SNs, to model a network utility function as the inverse of the energy consumption of SN sending and the energy consumption required for UAV to collect data, with the goal of maximizing network utility, to achieve joint optimization of UAV trajectories and user association strategies.
[0005] In order to achieve the above object, the present invention provides the following technical solutions:
[0006] A method for collecting drone data comprises the following steps:
[0007] S1: Modeling UAV model;
[0008] S2: Modeling SN model;
[0009] S3: Modeling channel model;
[0010] S4: Determine SN clustering strategy based on affinity propagation algorithm;
[0011] S5: Modeling network utility functions;
[0012] S6: Modeling resource scheduling and bandwidth allocation constraints;
[0013] S7: Modeling system states, actions, and output rewards;
[0014] S8: Model and train a double-delay deep deterministic policy gradient network;
[0015] S9: Determine the UAV data collection strategy based on the double-delay deep deterministic policy gradient algorithm.
[0016] Furthermore, the UAV modeling described in step S1 specifically includes: dividing the system time into time slots of equal size, letting T represent the total number of time slots, τ representing the length of the time slot, and assuming that the position of the UAV is fixed in a time slot; the UAV starts from the initial position, collects data from all SNs, and then returns to the initial position, letting H be the flight height of the UAV, and the position of the UAV in time slot t is The maximum speed of the UAV is v max ;v t ∈[0,v max ] represents the flying speed of UAV in time slot t; let B0 represent the available bandwidth of UAV.
[0017] Furthermore, the SN modeling described in step S2 specifically includes: m represents the mth SN, x m Indicates SN m The packet size, SN m The location is 1≤m≤M, M is the number of SNs. SNs collect data from the environment and transmit them to the UAV through orthogonal frequency division multiple access technology. In the time slot t, at most m0 SNs can upload data to the UAV at the same time.
[0018] Furthermore, the modeling of the channel model in step S3 specifically includes: modeling time slot t, SN m The channel model between UAV is as follows: Where a is the path loss coefficient, ρ0 is the reference channel gain at a distance of 1m, Indicates time slot t, UAV and SN m The distance between It is modeled as a Gaussian random variable, and e∈{LoS,NLoS} is the propagation parameter.
[0019] Furthermore, the SN clustering strategy based on the affinity propagation algorithm described in step S4 specifically includes: using the similarity between SNs to divide them into K clusters; setting the binary indicator matrix ρ = [ρ m,k ] M×M , where ρ m,k ∈{0,1}, if ρ m,k =1, indicating SN k SN m The cluster center of m,k =0;
[0020] Based on the optimization problem The cluster centers are optimized, where κ is a non-negative constant that represents the ability of a node to serve as a cluster center;
[0021] The constraints for modeling SNs clustering are:
[0022] ① Indicates that a SN can only be associated with one cluster center;
[0023] ② It means that a cluster contains at most m0 SNs;
[0024] Let S = (s m,k ) M×M is the similarity matrix, where s m,k Indicates SN m and SN k The similarity between them is modeled as:
[0025]
[0026] Let Ψ=(ψ m,k ) M×M is the attraction matrix, where ψ m,k Indicates SN k Become a covered SN m The fitness of the cluster center; let is the attribution matrix, where SN m Select SN k as the fitness of its cluster center; given SN k , initialize ψ m,k =0, Assume n is the current iteration number, They are respectively m,k , The nth iteration value of ψ m,k and The update formula is as follows:
[0027]
[0028]
[0029] Repeat the update steps until the algorithm converges and obtains the cluster center. Then the clustering strategy of SN is determined as:
[0030] Further, in step S5, the modeling network utility function is:
[0031]
[0032] Where ζ and ω are the weight coefficients of SN transmission energy consumption and UAV propulsion energy consumption, respectively; is the transmission energy consumption of SN in time slot t, E m,t Indicates SN m The energy consumption in time slot t is calculated as: E m,t =α m,t p m,t D m,t , where α m,t ∈{0,1}, if α m,t =1, indicating SN m Upload data to UAV at time slot t, otherwise, α m,t =0; p m,t Indicates SN m The transmission power at time slot t, D m,t Represents the latency of uploading data, which is modeled as: R m,t Indicates SN m The transmission rate in time slot t is modeled as Among them B m,t SN m The bandwidth allocated in time slot t, σ 2 is the noise power;
[0033] represents the energy consumption of UAV in time slot t, which is modeled as Where l0 is the given threshold value, is the hovering energy consumption of the UAV, modeled as Where P h is the hovering energy consumption; represents the energy consumption of the UAV during flight in time slot t, and the calculation formula is:
[0034]
[0035] Where P0 and P i are constants, representing the blade profile power and the induced power in the hovering state, U tip is the tip speed of the rotor blade, v0 is the average rotor induced speed when the UAV is hovering, d0 and s are the fuselage drag ratio and rotor reliability, respectively, ρ and Λ are the air density and rotor disk area, respectively.
[0036] Furthermore, the modeling resource scheduling and bandwidth allocation constraint conditions in step S6 specifically include:
[0037]
[0038]
[0039] Furthermore, in step S7, the modeling system state, action and output reward specifically include: the state of the modeling system at time slot t where h t ={h 1,t ,h 2,t ,...,h M,t} represents the set of channel gains between UAV and SN; modeling time slot t, action a t =<θ t ,v t ,B t ,C t >, where θ t ∈(0,2π] is the flight direction of the UAV, B t = {B 1,t ,B 2,t ,…,B M,t}, C t ={α 1,t ,α 2,t ,…,α M,t}; Modeling time slot t, output reward of UAV
[0040] Further, in step S8, the double-delay deep deterministic policy gradient network is modeled and trained, specifically including: initializing the discount factor γ, the update rate υ; the online policy network parameters of the actor network and the two comment networks are θ1 and θ2, copy the online policy network parameters to the target network, the parameters are θ′1 and θ′2; Initialize the experience replay buffer Initialize a small sample set And the system status s t Initialize; for U, according to the strategy Action a t Applied to the system environment, the reward value r is obtained t (s t ,a t ) and the state s at the next moment t+1 , and (s t ,a t ,r t ,s t+1 ) is stored in the experience replay buffer from Draw a sample from pass Calculate s t+1 The value of the action corresponding to the state The target value is Based on minimizing the loss function Update the review network; use the policy gradient update formula Update its actor network; update the parameters of the online policy network and target network through the soft update algorithm, specifically: θ′ i ←υθ i,t +(1-υ)θ′ i,t ,i=1,2.
[0041] Furthermore, in step S9, the UAV data collection strategy is determined based on the double-delay deep deterministic policy gradient algorithm, specifically including: inputting environmental observation values into the network, and determining the UAV flight trajectory, SN bandwidth allocation and resource scheduling strategy according to the output of the online policy network.
[0042] The beneficial effect of the present invention is that the present invention jointly optimizes the UAV trajectory and sensor node association strategy while comprehensively considering the bandwidth allocation and resource scheduling restrictions of the SN to maximize the network utility.
[0043] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:
[0045] Figure 1 Schematic diagram of the drone data collection scenario;
[0046] Figure 2 Flowchart of the UAV data collection method based on the double-delay deep deterministic policy gradient algorithm. DETAILED DESCRIPTION
[0047] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.
[0048] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.
[0049] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.
[0050] See also Figure 1-2 , Figure 1 Schematic diagram of the drone data collection scenario, such as Figure 1 As shown, there is a drone and multiple sensor nodes in the network. The sensor nodes collect environmental information and collect data through drones. At the same time, drones jointly optimize trajectories and node association strategies based on maximizing network utility.
[0051] Figure 2 Flow chart of the drone data collection method of the present invention, as shown in Figure 2 As shown, the method specifically comprises the following steps:
[0052] 1) Modeling UAV
[0053] The system time is divided into time slots of equal size. Let T represent the total number of time slots, τ represent the length of the time slot, and assume that the position of the UAV is fixed in a time slot. The UAV starts from the initial position, collects data from all SNs, and then returns to the initial position. Let H be the flight altitude of the UAV. The position of the UAV in time slot t is The maximum speed of the UAV is v max ;v t ∈[0,v max ] represents the flying speed of UAV in time slot t; let B0 represent the available bandwidth of UAV.
[0054] 2) Modeling SN model
[0055] SN m represents the mth SN, x m Indicates SN mThe packet size, SN m The location is 1≤m≤M, M is the number of SNs. SNs collect data from the environment and transmit them to the UAV through orthogonal frequency division multiple access technology. In the time slot t, at most m0 SNs can upload data to the UAV at the same time.
[0056] 3) Modeling channel model
[0057] Modeling time slot t, SN m The channel model between UAV is as follows: Where a is the path loss coefficient, ρ0 is the reference channel gain at a distance of 1m, Indicates time slot t, UAV and SN m The distance between It is modeled as a Gaussian random variable, and e∈{LoS,NLoS} is the propagation parameter.
[0058] 4) Determine SN clustering strategy based on affinity propagation algorithm
[0059] Use the similarity between SNs to divide them into K clusters; let the binary indicator matrix ρ = [ρ m,k ] M×M , where ρ m,k ∈{0,1}, if ρ m,k =1, indicating SN k SN m The cluster center of m,k =0;
[0060] Based on the optimization problem The cluster centers are optimized, where κ is a non-negative constant that represents the ability of a node to serve as a cluster center;
[0061] The constraints for modeling SNs clustering are:
[0062] ① Indicates that a SN can only be associated with one cluster center;
[0063] ② It means that a cluster contains at most m0 SNs;
[0064] Let S = (s m,k ) M×M is the similarity matrix, where s m,k Indicates SN m and SN k The similarity between them is modeled as:
[0065]
[0066] Let Ψ=(ψm,k ) M×M is the attraction matrix, where ψ m,k Indicates SN k Become a covered SN m The fitness of the cluster center; let is the attribution matrix, where It's SN m Select SN k as the fitness of its cluster center; given SN k , initialize ψ m,k =0, Assume n is the current iteration number, They are respectively m,k , The nth iteration value of ψ m,k and The update formula is as follows:
[0067]
[0068]
[0069] Repeat the above update steps until the algorithm converges and obtains the cluster center. Then the clustering strategy of SN is determined as:
[0070] 5) Modeling network utility function
[0071] Network Utility Where ζ and ω are the weight coefficients of SN transmission energy consumption and UAV propulsion energy consumption, respectively;
[0072] in is the transmission energy consumption of SN in time slot t, E m,t Indicates SN m The energy consumption in time slot t is calculated as: E m,t =α m,t p m,t D m,t , where α m,t ∈{0,1}, if α m,t =1, indicating SN m Upload data to UAV at time slot t, otherwise, α m,t =0; p m,t Indicates SN m The transmission power at time slot t, D m,t Represents the latency of uploading data, which is modeled as: R m,t Indicates SN m The transmission rate in time slot t is modeled as Among them Bm,t SN m The bandwidth allocated in time slot t, σ 2 is the noise power;
[0073] represents the energy consumption of UAV in time slot t, which is modeled as Where l0 is the given threshold value, is the hovering energy consumption of the UAV, modeled as Where P h is the hovering energy consumption; represents the energy consumption of the UAV during flight in time slot t, and the calculation formula is:
[0074] Where P0 and P i are constants, representing the blade profile power and the induced power in the hovering state, U tip is the tip speed of the rotor blade, v0 is the average rotor induced speed when the UAV is hovering, d0 and s are the fuselage drag ratio and rotor reliability, respectively, ρ and Λ are the air density and rotor disk area, respectively.
[0075] 6) Modeling resource scheduling and bandwidth allocation constraints
[0076] ①
[0077] ②
[0078] 7) Modeling system states, actions, and output rewards
[0079] The state of the modeled system at time slot t where h t ={h 1,t ,h 2,t ,...,h M,t} represents the set of channel gains between UAV and SN; modeling time slot t, action a t =<θ t ,v t ,B t ,C t >, where θ t ∈(0,2π] is the flight direction of the UAV, B t = {B 1,t ,B 2,t ,…,B M,t}, C t ={α 1,t ,α 2,t ,…,α M,t}; Modeling time slot t, output reward of UAV
[0080] 8) Modeling and training a double-delay deep deterministic policy gradient network
[0081] Initialize the discount factor γ and update rate υ; the online policy network parameters of the actor network and the two critic networks are θ1 and θ2, copy the online policy network parameters to the target network, the parameters are θ′1 and θ′2; Initialize the experience replay buffer Initialize a small sample set And the system status s t Initialize; for U, according to the strategy Action a t Applied to the system environment, the reward value r is obtained t (s t ,a t ) and the state s at the next moment t+1 , and (s t ,a t ,r t ,s t+1 ) is stored in the experience replay buffer from Draw a sample from pass Calculate s t+1 The value of the action corresponding to the state The target value is Based on minimizing the loss function Update the review network; use the policy gradient update formula Update its actor network; update the parameters of the online policy network and target network through the soft update algorithm, specifically: θ′ i ←υθ i,t +(1-υ)θ′ i,t ,i=1,2.
[0082] 9) Input the environmental observation values into the network, and determine the UAV flight trajectory, SN bandwidth allocation and resource scheduling strategy based on the output of the online strategy network.
[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.
Claims
1. A method for collecting drone data, characterized in that: The following steps are involved: S1: Modeling UAV model; S2: Modeling SN model; S3: Modeling channel model; S4: Determine SN clustering strategy based on affinity propagation algorithm; S5: Modeling network utility functions; S6: Model resource scheduling and bandwidth allocation constraints, including: T represents the total number of time slots, t represents the current time slot, M represents the number of SNs, m represents the sequence number of the mth SN, and m0 represents that within time slot t, at most m0 SNs can upload data to the UAV at the same time; α m,t ∈{0,1}, if α m,t =1, indicating SN m Upload data to UAV at time slot t, otherwise, α m,t =0; B m,t SN m The bandwidth allocated in time slot t; B0 represents the available bandwidth of the UAV; S7: Modeling system states, actions, and output rewards, specifically including: Modeling the state of the system at time slot t in represents the position of the UAV at time slot t, h t ={h 1,t ,h 2,t ,…,h M,t } represents the set of channel gains between UAV and SN; modeling time slot t, action a t =<θ t ,v t ,B t ,C t >, where θ t ∈(0,2π] is the flight direction of the UAV, v t represents the flight speed of the UAV at time slot t, B t = {B 1,t ,B 2,t ,…,B M,t }, C t ={α 1,t ,α 2,t ,…,α M,t }; Modeling time slot t, output reward of UAV in represents the energy consumption of UAV in time slot t, is the transmission energy consumption of SN at time slot t, ζ and ω are the weight coefficients of SN transmission energy consumption and UAV propulsion energy consumption, respectively; S8: Model and train a double-delay deep deterministic policy gradient network, including: initializing the discount factor γ, the update rate υ; the online policy network parameters of the actor network and the two comment networks are θ1 and θ2, copy the online policy network parameters to the target network, the parameters are θ1′ and θ′2; Initialize the experience replay buffer Initialize a small sample set And the system status s t Initialize; for U, according to strategy a t =μ(s t )+n t , Action a t Applied to the system environment, the reward value r is obtained t (s t ,a t ) and the state st at the next moment +1 , and (s t ,a t ,r t ,s t+1 ) is stored in the experience replay buffer from Draw a sample from pass Calculate s t+1 The value of the action corresponding to the state The target value is Based on minimizing the loss function Update the review network; use the policy gradient update formula Update its actor network; update the parameters of the online policy network and target network through the soft update algorithm, specifically: θ i ′←υθ i,t +(1-υ)θ′ i,t ,i=1,2; S9: Determine the UAV data collection strategy based on the double-delay deep deterministic policy gradient algorithm.
2. The method for collecting drone data according to claim 1, characterized in that: The UAV modeling described in step S1 specifically includes: dividing the system time into time slots of equal size, let T represent the total number of time slots, τ represent the length of the time slot, and assume that the position of the UAV is fixed in a time slot; the UAV starts from the initial position, collects data from all SNs, and then returns to the initial position, let H be the flight altitude of the UAV, and the position of the UAV in time slot t is The maximum speed of the UAV is v max ;v t ∈[0,v max ] represents the flying speed of UAV in time slot t; let B0 represent the available bandwidth of UAV.
3. The method for collecting drone data according to claim 1, characterized in that: The SN modeling described in step S2 specifically includes: m represents the mth SN, x m Indicates SN m The packet size, SN m The location is 1≤m≤M, M is the number of SNs. SNs collect data from the environment and transmit them to the UAV through orthogonal frequency division multiple access technology. In the time slot t, at most m0 SNs can upload data to the UAV at the same time.
4. The method for collecting drone data according to claim 1, characterized in that: The channel modeling described in step S3 specifically includes: modeling time slot t, SN m The channel model between UAV is as follows: Where a is the path loss coefficient, ρ0 is the reference channel gain at a distance of 1m, Indicates time slot t, UAV and SN m The distance between It is modeled as a Gaussian random variable, and e∈{LoS,NLoS} is the propagation parameter.
5. The method for collecting drone data according to claim 1, characterized in that: The SN clustering strategy based on the affinity propagation algorithm in step S4 specifically includes: dividing the SNs into K clusters based on their similarities; setting the binary indicator matrix ρ = [ρ m,k ] M×M , where ρ m,k ∈{0,1}, if ρ m,k =1, indicating SN k SN m The cluster center of m,k =0; Based on the optimization problem The cluster centers are optimized, where κ is a non-negative constant that represents the ability of a node to serve as a cluster center; The constraints for modeling SNs clustering are: ① Indicates that a SN can only be associated with one cluster center; ② It means that a cluster contains at most m0 SNs; Let S = (s m,k ) M×M is the similarity matrix, where s m,k Indicates SN m and SN k The similarity between them is modeled as: Let Ψ=(ψ m,k ) M×M is the attraction matrix, where ψ m,k Indicates SN k Become a covered SN m The fitness of the cluster center; let is the attribution matrix, where SN m Select SN k as the fitness of its cluster center; given SN k , initialize ψ m,k =0, Assume n is the current iteration number, They are respectively m,k , The nth iteration value of ψ m,k and The update formula is as follows: Repeat the update steps until the algorithm converges and obtains the cluster center. Then the clustering strategy of SN is determined as:
6. The method for collecting drone data according to claim 1, characterized in that: In step S5, the modeling network utility function is: Where ζ and ω are the weight coefficients of SN transmission energy consumption and UAV propulsion energy consumption, respectively; is the transmission energy consumption of SN in time slot t, E m,t Indicates SN m The energy consumption in time slot t is calculated as: E m,t =α m,t p m,t D m,t , where α m,t ∈{0,1}, if α m,t =1, indicating SN m Upload data to UAV at time slot t, otherwise, α m,t =0; p m,t Indicates SN m The transmission power at time slot t, D m,t Represents the latency of uploading data, which is modeled as: R m,t Indicates SN m The transmission rate in time slot t is modeled as Among them B m,t SN m The bandwidth allocated in time slot t, σ 2 is the noise power; represents the energy consumption of UAV in time slot t, which is modeled as Where l0 is the given threshold value, is the hovering energy consumption of the UAV, modeled as Where P h is the hovering energy consumption; represents the energy consumption of the UAV during flight in time slot t, and the calculation formula is: Where P0 and P i are constants, representing the blade profile power and the induced power in the hovering state, U tip is the tip speed of the rotor blade, v0 is the average rotor induced speed when the UAV is hovering, d0 and s are the fuselage drag ratio and rotor reliability, respectively, ρ and Λ are the air density and rotor disk area, respectively.
7. The method for collecting drone data according to claim 1, characterized in that: In step S9, the UAV data collection strategy is determined based on the double-delay deep deterministic policy gradient algorithm, specifically including: inputting environmental observation values into the network, and determining the UAV flight trajectory, SN bandwidth allocation and resource scheduling strategy according to the output of the online policy network.
Citation Information
Patent Citations
Dynamic data collection method based on unmanned aerial vehicle
CN112333648A
A communication network hotspot area capacity enhancement-oriented multi-air base station deployment method
CN113873434A
