A method for optimizing energy efficiency of wireless sensor networks based on drone assistance

Abstract

The present invention discloses a method for optimizing energy efficiency of wireless sensor networks assisted by drones. The method collects network status through an old routing scheme, inputs the status into an intelligent agent's decision network, and makes decisions on hovering nodes. Based on the positions of the hovering nodes, a new multi-hop routing scheme is determined for the network, and the new routing scheme is distributed through the old routing scheme. After all sensor nodes receive the new routing scheme, all sensors transmit the collected data to the hovering nodes according to the new routing scheme. The drones then fly to the hovering nodes to receive the data, thereby completing data collection for the entire wireless sensor network. The method takes into account that node energy consumption varies with the amount of forwarded data, so the data collection scheme is determined online. After the relative change in node energy reaches a certain level, the drones need to re-determine the next hovering node and a new multi-hop routing scheme based on the current network status. This maximizes network survival time through the cooperation between the drones and sensor nodes.

Description

Technical Field

[0001] The present invention belongs to the field of communication technology, and more specifically, relates to a method for optimizing energy efficiency of wireless sensor networks based on the assistance of drones. Background Art

[0002] With the continuous development of the Internet of Things (IoT), wireless sensor networks (WSNs), as one of the key technologies of the IoT, are widely deployed in various scenarios such as environmental monitoring, industrial control, and smart cities. In most scenarios, WSN sensor nodes are deployed temporarily and powered by batteries with limited energy, which are often difficult to recharge or replace. Therefore, given the limited energy of sensor nodes in wireless sensor networks, maximizing the survival time of wireless sensor networks is particularly important.

[0003] A WSN typically consists of a power-connected sink node and multiple battery-powered sensor nodes. Data collected by the sensor nodes is wirelessly transmitted over a single or multiple hops to the sink node, which then forwards it to the core network server for further processing. Data forwarding by sensor nodes accounts for a significant portion of total energy consumption, making the energy consumption of this stage of data forwarding a concern. Since most WSNs use multi-hop routing schemes for data collection, and sensor nodes closer to the sink node often forward more data than those farther away, sensor nodes closer to the sink node consume more energy faster, resulting in uneven energy distribution and premature failure of the WSN.

[0004] Unmanned aerial vehicles (UAVs) offer a new solution to the problem of uneven energy distribution in WSNs leading to premature failure. As flying data collectors, UAVs are highly flexible and can move quickly and unimpeded. When energy distribution in a WSN is uneven, UAVs can fly to areas with high energy levels among sensor nodes to collect data across the entire network, balancing the energy consumption rates of individual sensor nodes. Using UAVs to assist WSNs in collecting sensor data is a typical application for extending network lifespan.

[0005] The algorithm design for drone-assisted WSN data collection needs to consider two key issues: the drone's path planning problem and the multi-hop routing problem of sensor nodes. The drone's path planning problem refers to the fact that as network data is continuously collected, the network status is constantly changing. The drone needs to go to the next location to collect network data at the appropriate time. How to decide on the next location is a key issue. The multi-hop routing problem of sensor nodes refers to the problem of designing a multi-hop routing solution when the drone's data collection location is known, so that all sensor nodes can quickly transmit data to the drone while consuming less energy. How to determine the next drone location and how to design a multi-hop routing solution that continuously adjusts as the network status changes to maximize network survival time are issues that need to be solved. Summary of the Invention

[0006] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide a method for optimizing the energy efficiency of wireless sensor networks based on drone assistance. In the scenario where the energy of sensor nodes in WSN is limited, the network survival time can be maximized through the mutual cooperation between drones and sensor nodes.

[0007] In order to achieve the above-mentioned object of the invention, the present invention is based on a method for optimizing energy efficiency of a wireless sensor network assisted by a drone, which is characterized by comprising the following steps:

[0008] (1) In a simulation environment, train the intelligent agent for drone hovering node decision-making

[0009] According to the actual deployment, establish A battery-powered sensor nodes n1,…,n in the simulation environment. A and a wireless sensor network with one drone as the sink node;

[0010] For sensor node n i ,i=1,…,A, determine all other sensor nodes within its communication range as its neighbor nodes, and build a neighbor node list in, represents sensor node n i The cth neighbor node of i |,|nbr i | represents the number of neighbor nodes;

[0011] An intelligent agent is deployed on the UAV to make decisions about hovering nodes, where the hovering nodes are sensor nodes that the UAV will hover over to collect data from the entire wireless sensor network.

[0012] The agent is trained using a deep reinforcement learning algorithm based on actor-critic:

[0013] 1.1) Select any sensor node as the hovering node, and use the distance between sensor nodes as weight according to the deployment location of the sensor nodes and the neighbor relationship between the sensor nodes. Use the Kruskal algorithm to calculate the minimum spanning tree of the wireless sensor network, and use the hovering node as the root node in the spanning tree. Use the breadth-first-search algorithm (BFS) to calculate the routing information of each sensor node;

[0014] 1.2) For the different data that the sensor nodes need to collect, we design corresponding probability distributions based on the existing prior knowledge to simulate the amount of data collected by the sensor nodes in the real environment. We then send the collected data to the hovering nodes according to the routing information every α seconds. The hovering nodes transmit data to the drone while it is hovering. At the same time, we simulate the energy consumption of the sensor nodes.

[0015] 1.3) Every time the sensor node performs β rounds of transmission, the drone makes a decision on the next hovering node and generates a new routing solution. The decision-making process is as follows:

[0016] 1.3.1) The drone decides the next hovering node

[0017] 1.3.1.1) Each sensor node n i When the β-round transmission is completed, the remaining energy information is sent to the UAV through the existing routing, and the UAV then normalizes the remaining energy information to obtain the remaining energy value W i , and then get the residual energy vector of all sensor nodes

[0018] 1.3.1.2) The drone obtains the position vector of the sensor node based on the position of each sensor node in and Corresponding to sensor node n i The horizontal and vertical coordinates are normalized in a fixed coordinate system;

[0019] 1.3.1.3) The drone will state vector Input into the decision network of the intelligent agent and calculate the output probability vector where p i In the hovering node decision, the drone selects sensor node n i The probability of being the next hovering node;

[0020] 1.3.1.4) The drone randomly generates a floating point number in the range of (0,1], and the floating point number will be located in the probability vector The jth interval of the probability cumulative distribution function corresponds to The j-th probability value indicates that the UAV selects sensor node n j as the next hover node;

[0021] 1.3.2) The drone generates a new routing plan

[0022] 1.3.2.1) The drone uses the Energy-Balanced Routing Protocol (EBRP) algorithm to calculate the number of nodes n. i , based on its neighbor node list Calculate the mixed potential energy field list in, Represents node n i Its neighboring nodes The mixed potential energy field, The value of indicates the selection of neighbor nodes As the preference of the parent node, the larger the value, the more biased the selection;

[0023] 1.3.2.2) The drone detects each sensor node n according to its location. i Calculate the next hovering node n j Distance, sort in descending order according to distance to get a node list in, Represents a node list The i-th sensor node;

[0024] 1.3.2.3) The drone maintains an edge set E, which is initially empty and represents the edges in the spanning tree generated by the wireless sensor network, and the root node of the spanning tree is

[0025] 1.3.2.4) The drone traverses the node list from the beginning From the sensor node Traverse to the sensor nodes in sequence Select the parent node of each sensor node from the farthest to the closest distance to the hovering node, and guide the sensor node to transmit data to the hovering node, let i = 1;

[0026] 1.3.2.4.1) For sensor nodes If i is equal to N, execute step 1.3.3); if i is not equal to N, execute step 1.3.2.4.2);

[0027] 1.3.2.4.2) At this time, the sensor node Corresponding sensor node n k , for the mixed potential field list U k Sort in descending order to get a list in represents sensor node n k and the cth neighbor node after descending sorting The mixed potential energy field;

[0028] 1.3.2.4.3) The drone traverses the list again That is, from Traverse in sequence Give priority to the neighbor node with large preference as the parent node, let c = 1;

[0029] 1.3.2.4.3.1) For sensor nodes Check the corresponding edges Will it form a ring after adding edge set E? If so, execute step 1.3.2.4.3.2). If not, add edge set E. Add edge set E and execute steps 1.3.2.4.4);

[0030] 1.3.2.4.3.2) If c is equal to |nbr i |, then use the Minimum Directed Spanning Tree (MDST) algorithm to calculate the minimum dendrogram, let the edge set E equal to the set of all edges in the minimum dendrogram, and execute step 1.3.3). If c is not equal to |nbr i |, then set c = c + 1 and return to step 1.3.2.4.3.1);

[0031] 1.3.2.4.4) Set i = i + 1 and return to step 1.3.2.4.1);

[0032] 1.3.2.5) The drone obtains a spanning tree based on the edge set E and places the sensor node n in the spanning tree. j As the root node, the BFS algorithm is used to calculate the new routing solution for each sensor node. j Then wait for the drone to hover and then transmit the data to the drone;

[0033] 1.3.3) The drone packages the new routing plan corresponding to each sensor node and sends it to each sensor node according to the old routing plan. After receiving it, each sensor transmits data according to the new routing plan. The drone then flies to the next hovering node and hovers to collect data.

[0034] 1.4) Continuously execute step 1.3) until at least one sensor node runs out of energy, at which point the wireless sensor network is paralyzed. Then, train the agent: the agent's decision network is an actor network, and the state vector of its hovering node at the decision moment is As the input of the actor network and the critic network, the actor-critic deep reinforcement learning algorithm is used for training. The reward function used in the training process is calculated based on the survival time of the wireless sensor network and the overall energy consumption of the sensor nodes. The specific calculation formula is:

[0035]

[0036] Among them, R t R represents the reward function value of the tth hovering node decision; E It indicates that the setting is made based on the overall energy consumption of the sensor node between the tth hovering node decision and the t-1th hovering node decision. The less the overall energy consumption, the better R E The larger the value, the T Indicates the reward when the network is paralyzed. It is set according to the network survival time. The longer the network survival time, the higher the reward. T The larger the value;

[0037] 1.5) Repeat steps 1.1) to 1.4) and continuously update the network weights of the actor and critic networks until convergence;

[0038] (2) Deploy UAVs and wireless sensor networks into real-world environments

[0039] 2.1) Select any node n according to the method in step 1.1) i It is a hovering node and calculates the routing information of each sensor node;

[0040] 2.2) Write the deployment location, neighbor information, and routing information of the sensor nodes into the configuration files of each sensor node and the UAV, and deploy an agent on the UAV to make decisions about hovering nodes. Its decision network is the decision network of the agent trained in the simulation environment in step (1);

[0041] 2.3) Deploy each sensor node in the actual environment according to the deployment location, and the drone hovers over the sensor node n i above;

[0042] (3) All sensor nodes will continuously detect the environment, collect data, and send the collected data to the hovering node according to the routing information every α seconds. The hovering node transmits all data to the drone when it is hovering;

[0043] (4) Every time the sensor node performs β rounds of transmission, the UAV decides the next hovering node according to the method in step 1.3.1), generates a new routing plan according to the method in step 1.3.2), and then sends it to each sensor node according to step 1.3.3) and flies to the next hovering node to hover and collect data.

[0044] The object of the invention of the present invention is achieved like this:

[0045] The present invention's drone-assisted wireless sensor network energy efficiency optimization method includes two steps: training an agent deployed on a drone in a simulated environment; and, in a real-world environment, using the agent's decision network to determine the next hovering node and generate a new routing plan. Specifically, the network state is collected using the old routing plan, which is then input into the agent's decision network to make decisions about hovering nodes. A new multi-hop routing plan is determined based on the location of the hovering nodes and distributed using the old routing plan. After all sensor nodes receive the new routing plan, all sensors transmit collected data to the hovering nodes according to the new routing plan. The drone then flies to the hovering nodes to receive the data, completing data collection for the entire wireless sensor network. The present invention considers that node energy consumption varies with the amount of forwarded data, so the data collection plan is determined online. After the relative change in node energy reaches a certain level, the drone re-determines the next hovering node and a new multi-hop routing plan based on the current network state. This collaborative effort between the drone and sensor nodes maximizes network uptime.

[0046] At the same time, the UAV-assisted wireless sensor network energy efficiency optimization method of the present invention also has the following beneficial effects:

[0047] 1. This invention implements a drone-assisted wireless sensor network data collection solution. The drone can hover above all sensor nodes and collect data from the entire WSN network. Compared with traditional wireless sensor network data collection solutions with fixed sink node locations, this solution is more flexible and can better adapt to the relative changes in sensor node energy.

[0048] 2. The present invention can avoid transmitting redundant residual energy information between sensor nodes. The drone aggregates and distributes the information, thereby reducing the energy consumption of the sensor nodes and making the energy utilization of the sensor more efficient.

[0049] 3. This paper designs a data collection scheme during normal network operation. This scheme can adjust the drone hovering collection position and multi-hop routing scheme in real time according to the energy changes of sensor nodes in the wireless sensor network, thereby maximizing the network survival time.

[0050] 4. The present invention uses deep reinforcement learning technology to make decisions on drone hovering nodes, that is, the design of drone movement plans, so that it is compatible with the multi-hop routing generation plan and jointly maximizes the survival time of the network. Compared with heuristic solutions, it can make decisions more quickly and effectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flowchart of a specific implementation method of the UAV-assisted wireless sensor network energy efficiency optimization method of the present invention;

[0052] Figure 2 It is a schematic diagram of the deployment location of sensor nodes;

[0053] Figure 3 is the sensor node n 15 Schematic diagram of the corresponding neighbor set;

[0054] Figure 4 This is a flowchart for training an agent;

[0055] Figure 5 This is a flowchart of the drone making decisions about the next hovering node and generating a new routing solution.

[0056] Figure 6 This is the neural network architecture diagram of the agent's decision network;

[0057] Figure 7 Schematic diagram of the hovering node positions and multi-hop routing solutions decided and generated at different stages after deployment, where (a), (b), and (c) are the hovering node positions and routing solutions generated at initialization, the 100th decision, and the 200th decision, respectively. DETAILED DESCRIPTION

[0058] The following describes the specific embodiments of the present invention in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when detailed descriptions of known functions and designs may dilute the main content of the present invention, such descriptions will be omitted here.

[0059] Figure 1 This is a flowchart of a specific implementation method of the wireless sensor network energy efficiency optimization method assisted by drones of the present invention.

[0060] In this embodiment, if Figure 1 As shown, the energy efficiency optimization method of wireless sensor networks based on drone assistance of the present invention includes the following steps:

[0061] Step S1: In the simulation environment, train the intelligent agent for drone hovering node decision-making

[0062] According to the actual deployment, establish A battery-powered sensor nodes n1,…,n in the simulation environment. A and a wireless sensor network with one drone as the sink node.

[0063] For sensor node n i ,i=1,…,A, limited by the rated power, its communication range is R meters, so all other sensor nodes within its communication range R are determined as its neighbor nodes, and a neighbor node list is constructed in, represents sensor node n i The cth neighbor node of i |,|nbr i | represents the number of neighbor nodes.

[0064] An intelligent agent is deployed on the UAV to make decisions about hovering nodes, where the hovering nodes are sensor nodes above which the UAV will hover to collect data from the entire wireless sensor network.

[0065] In this embodiment, if Figure 2 As shown in the figure, the wireless sensor network has 20 battery-powered sensor nodes n1,…,n2, numbered 1-20. 20 , and are evenly distributed in a central circle with a radius of 100 meters.

[0066] The drone has sufficient energy to assist in completing data collection. The sensor node is limited by the rated power and its communication range is R = 100 meters. i ,i=1,…,20), define its neighbor list Right now m in sensor node n i All other sensor nodes within the communication range constitute its neighbor nodes. Represents a sensor node With sensor node n i The distance between them, c=1,…,|nbr i |,|nbr i | represents the number of neighbor nodes. Figure 3 As shown, the nodes in the dotted area with thick lines are sensor nodes n 15 Neighbor nodes, sensor nodes n 15 Neighbor list

[0067] In this embodiment, the drone's hovering altitude is h = 50 meters. All sensor nodes perform a transmission cycle every α = 600 seconds. That is, each sensor node collects data for α = 600 seconds and then sends the collected data to the hovering node. All sensor nodes have sufficient time to complete data transmission, and each sensor node has sufficient storage and transmission capacity to complete data transmission. At the same time, the drone has sufficient time to fly to the next hovering node and has sufficient energy for flight, hovering, and data transmission. After every β = 10 rounds of transmission, the intelligent agent's decision network will make a decision for the hovering node and generate a new routing plan based on the decision result.

[0068] The agent is trained using a deep reinforcement learning algorithm based on actor-critic, such as Figure 4 As shown, the following steps are included:

[0069] Step S1.1: Select any sensor node as the hovering node, and use the distance between sensor nodes as weight according to the deployment location of the sensor nodes and the neighbor relationship between the sensor nodes. Use the Kruskal algorithm to calculate the minimum spanning tree of the wireless sensor network, and use the hovering node as the root node in the spanning tree. Use the breadth-first-search algorithm (BFS) to calculate the routing information of each sensor node.

[0070] Step S1.2: For the different data that the sensor nodes need to collect, the corresponding probability distribution is designed based on the existing prior knowledge to simulate the amount of data collected by the sensor nodes in the real environment. The collected data is sent to the hovering node according to the routing information every time interval α = 600 seconds. The hovering node transmits data to the UAV when the UAV is hovering; at the same time, the energy consumption of the sensor node is simulated.

[0071] Step S1.3: After each β=10 rounds of transmission by the sensor node, the UAV makes a decision on the next hovering node and generates a new routing solution. The decision-making process is as follows: Figure 5 The specific steps are as follows:

[0072] Step S1.3.1: The drone decides the next hovering node

[0073] Step S1.3.1.1: Each sensor node n i When the β-round transmission is completed, the remaining energy information is sent to the drone through the existing routing, avoiding redundant information transmission between sensor nodes. The drone then normalizes the remaining energy information to obtain the remaining energy value W i , and then get the residual energy vector of all sensor nodes In this embodiment, the residual energy vector

[0074] Step S1.3.1.2: The UAV obtains the position vector of the sensor node based on the position of each sensor node in and Corresponding to sensor node n i The horizontal and vertical coordinates are normalized in a fixed coordinate system. In this embodiment, the position vector

[0075] Step S1.3.1.3: The drone will transform the state vector Input into the decision network of the intelligent agent and calculate the output probability vector where p i In the hovering node decision, the drone selects sensor node n i As the probability of the next hovering node. In this embodiment, the probability vector

[0076] The probability vector output by the decision network is:

[0077]

[0078] The corresponding cumulative distribution function vector is:

[0079] [0.4,0.4,0.4,0.5,0.5,0.5,0.6,0.6,0.6,0.7,0.7,0.7,0.8,0.8,0.8,0.8,0.9,0.9,0.9,1].

[0080] Step S1.3.1.4: The drone randomly generates a floating point number in the range (0,1], and the floating point number will be located in the probability vector The jth interval of the probability cumulative distribution function corresponds to The j-th probability value indicates that the UAV selects sensor node n j as the next hover node.

[0081] In this embodiment, the randomly generated floating point number is 0.43, which just falls into the 4th to 6th items of the cumulative distribution function vector (the starting subscript of the vector is 1), and the sensor node n4 is correspondingly selected as the next hovering node.

[0082] Step S1.3.2: The drone generates a new routing plan

[0083] Step S1.3.2.1: The drone uses the Energy-Balanced Routing Protocol (EBRP) algorithm to calculate the number of nodes n. i , based on its neighbor node list Calculate the mixed potential energy field list in, Represents node n i Its neighboring nodes The mixed potential energy field, The value of indicates the selection of neighbor nodes As the preference of the parent node, the larger the value, the more biased the selection. For the specific calculation process, please refer to the document "EBRP: energy-balanced routing protocol for data gathering in wireless sensor networks", Ren F, Zhang J, He T, et al. IEEE transactions on parallel and distributed systems, 2011, 22(12): 2108-2125.

[0084] Step S1.3.2.2: The UAV detects each sensor node n according to its location. i Calculate the next hovering node n j Distance, sort in descending order according to distance to get a node list in, Represents a node list The i-th sensor node.

[0085] Step S1.3.2.3: The drone maintains an edge set E, which is initially empty and represents the edges in the spanning tree generated by the wireless sensor network, and the root node of the spanning tree is

[0086] Step S1.3.2.4: The drone traverses the node list from the beginning From the sensor node Traverse to the sensor nodes in sequence Select the parent node of each sensor node, select the parent node of the node from far to close to the hovering node, guide the sensor node to transmit data to the hovering node, and let i=1.

[0087] Step S1.3.2.4.1: For sensor nodes If i is equal to N, execute step S1.3.3; if i is not equal to N, execute step S1.3.2.4.2.

[0088] Step S1.3.2.4.2: At this point, the sensor node Corresponding sensor node n k , for the mixed potential field list U k Sort in descending order to get a list in represents sensor node n k and the cth neighbor node after descending sorting mixed potential energy field.

[0089] Step S1.3.2.4.3: The drone traverses the list from the beginning That is, from Traverse in sequence Give priority to the neighbor node with large preference as the parent node, let c = 1;

[0090] Step S1.3.2.4.3.1: For sensor nodes Check the corresponding edges Will it form a ring after adding edge set E? If so, execute step S1.3.2.4.3.2. If not, add edge set E. Add edge set E and execute step S1.3.2.4.4.

[0091] Step S1.3.2.4.3.2: If c equals |nbr i |, then use the Minimum Directed Spanning Tree (MDST) algorithm to calculate the minimum dendrogram, let the edge set E equal to the set of all edges in the minimum dendrogram, and execute step S1.3.3. If c is not equal to |nbr i |, then set c=c+1 and return to step S1.3.2.4.3.1. The MDST algorithm can be found in the literature "Efficient algorithms for finding minimum spanning trees in undirected and directed graphs", Gabow HN, Galil Z, Spencer T, et al. Combinatorica, 1986, 6(2): 109-122.

[0092] Step S1.3.2.4.4: Let i=i+1 and return to step S1.3.2.4.4.

[0093] Step S1.3.2.5: The UAV obtains a spanning tree based on the edge set E and places the sensor node n in the spanning tree. j As the root node, the BFS algorithm is used to calculate the new routing solution for each sensor node.j Then wait for the drone to hover and then transmit the data to the drone.

[0094] Step S1.3.3: The UAV packages the new routing plan corresponding to each sensor node and sends it to each sensor node according to the old routing plan. After receiving it, each sensor transmits data according to the new routing plan. The UAV flies to the next hovering node and hovers to collect data.

[0095] Step S1.4: Continuously execute step S1.3 until at least one sensor node runs out of energy, at which point the wireless sensor network is paralyzed. Then, train the agent: the decision network of the agent is the actor network, and the state vector of its hovering node at the decision moment is As the input of the actor network and the critic network, the actor-critic deep reinforcement learning algorithm is used for training. The reward function used in the training process is calculated based on the survival time of the wireless sensor network and the overall energy consumption of the sensor nodes. The specific calculation formula is:

[0096]

[0097] Among them, R t R represents the reward function value of the tth hovering node decision; E It indicates that the setting is made based on the overall energy consumption of the sensor node between the tth hovering node decision and the t-1th hovering node decision. The less the overall energy consumption, the better R E The larger the value, the T Indicates the reward when the network is paralyzed. It is set according to the network survival time. The longer the network survival time, the higher the reward. T The larger the value.

[0098] In this embodiment, if Figure 6 As shown, the actor network, i.e. the decision network of the final deployed agent, consists of two fully connected layers activated by the ReLU (Rectified Linear Unit) function, with a width of 512, and finally passes through a Softmax layer to obtain the output probability vector

[0099] The critic network needs to evaluate the value of the current state to guide the learning of the actor network. Its input is consistent with that of the actor network. and Splice to get status Then it passes through three layers of fully connected layers activated by the ReLU (Rectified Linear Unit) function, with a width of 512 in this example, and finally passes through a linear layer to get the final output;

[0100] In this example, the Adam (Adaptive momentum) optimizer is used for training the actor and critic networks, and the learning rate used for the actor network is 1×10 -5 , the critic network uses a learning rate of 1×10 -4 To ensure training stability, we use GAE (Generalized Advantage Estimator) to estimate the advantage function when updating the critic network. To ensure the exploration intensity of the actor network and prevent it from prematurely falling into local optimal solutions, we add an entropy regularization term to the loss function, with an entropy regularization weight set to 0.01. The training of the actor and critic networks is well-known and will not be further elaborated here.

[0101] Step S1.5: Repeat steps S1.1 to S1.4, and continuously update the network weights of the actor and critic networks until convergence.

[0102] Step S2: Deploy the drone and wireless sensor network into the actual environment

[0103] Step S2.1: Select any node n according to the method of step S1.1 i It is a hovering node and calculates the routing information of each sensor node.

[0104] Step S2.2: Write the deployment location, neighbor information, and routing information of the sensor nodes into the configuration files of each sensor node and the UAV, and deploy an intelligent agent on the UAV to make decisions about hovering nodes. Its decision network is the decision network of the intelligent agent trained in the simulation environment in step S1.

[0105] Step S2.3: Deploy each sensor node in the actual environment according to the deployment location, and the drone hovers over the sensor node n i Above.

[0106] Step S3: All sensor nodes continuously monitor the environment and collect data. A determination is made as to whether any sensor nodes have exhausted their energy. If so, the wireless sensor network is disabled and sensor data collection ends. If all sensor nodes are still fully powered, each sensor node sends collected data to the hovering node according to the routing information at intervals of α seconds. The hovering node transmits all data to the drone while it is hovering.

[0107] Step S4: Every time the sensor node performs β rounds of transmission, it determines whether β rounds have been transmitted. If not, it returns to step S3. Otherwise, the drone obtains the network status and decides the next hovering node according to the method of step S1.3.1, and generates a new routing plan according to the method of step S1.3.2, and then sends it to each sensor node according to step S1.3.3 and flies to the next hovering node to hover and collect data, and returns to step S3.

[0108] Figure 7 This is a diagram of the hovering node locations and multi-hop routing solutions decided and generated at different stages after deployment. Figure 7 The percentage next to the node number indicates the remaining energy of the sensor node. Figure 7 It can be seen that the position of the hovering node is constantly changing, and the remaining energy of each sensor node is reduced, but relatively balanced, which maximizes the network survival time.

[0109] In order to illustrate the technical effect of the present invention, a specific example is used to verify the present invention. Figure 2 The wireless sensor network shown in Figure 1 is used for comparison. Two hovering node decision methods were selected, using the same multi-hop routing algorithm for the sensors. Comparison method 1 randomly selects from all nodes (Random); comparison method 2 selects the node with the lowest overall sensor node energy consumption in the next β = 10 rounds as the next hovering node (Greedy). The network survival times of the present invention and the two comparison methods were then compared. Table 1 shows a comparison of the network survival times of the present invention and the two comparison methods.

[0110] Routing Method random greedy The present invention Network survival time / number of rounds 1514 2179 2436

[0111] Table 1

[0112] As can be seen from Table 1, the present invention can make the wireless sensor network survive longer. Its network survival time is 1.6 times that of the Random method and 1.11 times that of the Greedy method, which proves that the present invention can maximize the network survival time.

[0113] Although the above describes the illustrative specific embodiments of the present invention to facilitate understanding of the present invention by those skilled in the art, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations using the concepts of the present invention are protected.

Claims

1. A method for optimizing energy efficiency of wireless sensor networks assisted by drones, comprising the following steps: (1) In a simulation environment, train the intelligent agent for drone hovering node decision-making According to the actual deployment, establish A battery-powered sensor nodes n1,…,n in the simulation environment. A and a wireless sensor network with one drone as the sink node; For sensor node n i ,i=1,…,A, determine all other sensor nodes within its communication range as its neighbor nodes, and build a neighbor node list in, represents sensor node n i The cth neighbor node of i |,|nbr i | represents the number of neighbor nodes; An intelligent agent is deployed on the UAV to make decisions about hovering nodes, where the hovering nodes are sensor nodes that the UAV will hover over to collect data from the entire wireless sensor network. The agent is trained using a deep reinforcement learning algorithm based on actor-critic: 1.1) Select any sensor node as the hovering node, and use the distance between sensor nodes as weight according to the deployment location of the sensor nodes and the neighbor relationship between the sensor nodes. Use the Kruskal algorithm to calculate the minimum spanning tree of the wireless sensor network, and use the hovering node as the root node in the spanning tree. Use the breadth-first-search algorithm (BFS) to calculate the routing information of each sensor node; 1.2) For the different data that the sensor nodes need to collect, we design corresponding probability distributions based on the existing prior knowledge to simulate the amount of data collected by the sensor nodes in the real environment. We then send the collected data to the hovering nodes according to the routing information every α seconds. The hovering nodes transmit data to the drone while it is hovering. At the same time, we simulate the energy consumption of the sensor nodes. 1.3) Every time the sensor node performs β rounds of transmission, the drone makes a decision on the next hovering node and generates a new routing solution. The decision-making process is as follows: 1.3.1) The drone decides the next hovering node 1.3.1.1) Each sensor node n i When the β-round transmission is completed, the remaining energy information is sent to the UAV through the existing routing, and the UAV then normalizes the remaining energy information to obtain the remaining energy value Wi , and then get the residual energy vector of all sensor nodes 1.3.1.2) The drone obtains the position vector of the sensor node based on the position of each sensor node in and Corresponding to sensor node n i The horizontal and vertical coordinates are normalized in a fixed coordinate system; 1.3.1.3) The drone will state vector Input into the decision network of the intelligent agent and calculate the output probability vector where p i In the hovering node decision, the drone selects sensor node n i The probability of being the next hovering node; 1.3.1.4) The drone randomly generates a floating point number in the range of (0,1], and the floating point number will be located in the probability vector The jth interval of the probability cumulative distribution function corresponds to The j-th probability value indicates that the UAV selects sensor node n j as the next hover node; 1.3.2) The drone generates a new routing plan 1.3.2.1) The drone uses the Energy-Balanced Routing Protocol (EBRP) algorithm to calculate the number of nodes n. i , based on its neighbor node list Calculate the mixed potential energy field list in, Represents node n i Its neighboring nodes The mixed potential energy field, The value of indicates the selection of neighbor nodes As the preference of the parent node, the larger the value, the more biased the selection; 1.3.2.2) The drone detects each sensor node n according to its location. i Calculate the next hovering node n j Distance, sort in descending order according to distance to get a node list in, Represents a node list The i-th sensor node; 1.3.2.3) The drone maintains an edge set E, which is initially empty and represents the edges in the spanning tree generated by the wireless sensor network, and the root node of the spanning tree is 1.3.2.4) The drone traverses the node list from the beginning From the sensor node Traverse to the sensor nodes in sequence Select the parent node of each sensor node from the farthest to the closest distance to the hovering node, and guide the sensor node to transmit data to the hovering node, let i = 1; 1.3.2.4.1) For sensor nodes If i is equal to N, execute step 1.3.3); if i is not equal to N, execute step 1.3.2.4.2); 1.3.2.4.2) At this time, the sensor node Corresponding sensor node n k , for the mixed potential field list U k Sort in descending order to get a list in represents sensor node n k and the cth neighbor node after descending sorting The mixed potential energy field; 1.3.2.4.3) The drone traverses the list again That is, from Traverse in sequence Give priority to the neighbor node with large preference as the parent node, let c = 1; 1.3.2.4.3.1) For sensor nodes Check the corresponding edges Will it form a ring after adding edge set E? If so, execute step 1.3.2.4.3.2). If not, add edge set E. Add edge set E and execute steps 1.3.2.4.4); 1.3.2.4.3.2) If c is equal to |nbr i |, then use the Minimum Directed Spanning Tree (MDST) algorithm to calculate the minimum dendrogram, let the edge set E equal to the set of all edges in the minimum dendrogram, and execute step 1.3.3). If c is not equal to |nbr i |, then set c = c + 1 and return to step 1.3.2.4.3.1); 1.3.2.4.4) Set i = i + 1 and return to step 1.3.2.4.1); 1.3.2.5) The drone obtains a spanning tree based on the edge set E and places the sensor node n in the spanning tree. j As the root node, the BFS algorithm is used to calculate the new routing solution for each sensor node. j Then wait for the drone to hover and then transmit the data to the drone; 1.3.3) The drone packages the new routing plan corresponding to each sensor node and sends it to each sensor node according to the old routing plan. After receiving it, each sensor transmits data according to the new routing plan. The drone then flies to the next hovering node and hovers to collect data. 1.4) Continuously execute step 1.3) until at least one sensor node runs out of energy, at which point the wireless sensor network is paralyzed. Then, train the agent: the agent's decision network is an actor network, and the state vector of its hovering node at the decision moment is As the input of the actor network and the critic network, the actor-critic deep reinforcement learning algorithm is used for training. The reward function used in the training process is calculated based on the survival time of the wireless sensor network and the overall energy consumption of the sensor nodes. The specific calculation formula is: in, R t R represents the reward function value of the tth hovering node decision; E It indicates that the setting is made based on the overall energy consumption of the sensor node between the tth hovering node decision and the t-1th hovering node decision. The less the overall energy consumption, the better R E The larger the value, the T Indicates the reward when the network is paralyzed. It is set according to the network survival time. The longer the network survival time, the higher the reward. T The larger the value; 1.5) Repeat steps 1.1) to 1.4) and continuously update the network weights of the actor and critic networks until convergence; (2) Deploy UAVs and wireless sensor networks into real-world environments 2.1) Select any node n according to the method in step 1.1) i It is a hovering node and calculates the routing information of each sensor node; 2.2) Write the deployment location, neighbor information, and routing information of the sensor nodes into the configuration files of each sensor node and the UAV, and deploy an agent on the UAV to make decisions about hovering nodes. Its decision network is the decision network of the agent trained in the simulation environment in step (1); 2.3) Deploy each sensor node in the actual environment according to the deployment location, and the drone hovers over the sensor node n i above; (3) All sensor nodes will continuously detect the environment, collect data, and send the collected data to the hovering node according to the routing information every α seconds. The hovering node transmits all data to the drone when it is hovering; (4) Every time the sensor node performs β rounds of transmission, the UAV decides the next hovering node according to the method in step 1.3.1), generates a new routing plan according to the method in step 1.3.2), and then sends it to each sensor node according to step 1.3.3) and flies to the next hovering node to hover and collect data.

Images