New energy grid-connected unmanned aerial vehicle auxiliary network average information age minimization strategy

Abstract

The invention discloses a new energy grid-connected unmanned aerial vehicle auxiliary network average information age minimization strategy, and provides an optimization strategy to minimize system average information age (AoI) for an unmanned aerial vehicle auxiliary wireless energy supply sensor network (WPCN) in a new energy grid-connected system. By deploying the wireless charging platform in the network, the charging round trip of the unmanned aerial vehicle is significantly reduced, the average AoI of the system is effectively reduced, and the operation efficiency is improved. In addition, a deep Q network (DQN)-based algorithm is adopted to optimize the trajectory and charging strategy of the unmanned aerial vehicle, and the task priority is allowed to be flexibly adjusted according to real-time data and energy requirements. A simulation result verifies the advantages of the strategy in the aspects of improving system performance and fairness between nodes, and powerful technical support is provided for real-time monitoring and management of a new energy grid-connected system.

Description

Technical Field

[0001] The present invention belongs to the field of UAV-assisted wireless energy supply communication technology, and specifically relates to a new energy grid-connected UAV-assisted network average information age minimization strategy. Background Art

[0002] With the rapid development of new energy technologies, especially in wind and solar power generation, real-time monitoring of widely distributed facilities to ensure efficiency and reliability is a major challenge.

[0003] New energy grid-connected systems require a high degree of operational accuracy to respond to environmental changes and changes in equipment status in real time. However, these facilities are often located in remote areas, making effective data monitoring and equipment management difficult.

[0004] Drones can cover a wide area, implement remote monitoring and data collection, especially with the support of wireless wireless power transmission (WPT) technology, they can provide a continuous supply of energy to ground sensors, thereby maintaining the continuous operation of sensors. Due to the high mobility and rapid deployment capabilities of drones, they are an ideal technology for real-time monitoring across a wide range of new energy facilities.

[0005] In the renewable energy grid-connected system, ensuring the real-time and integrity of data is crucial to maintaining the continuity of the system. Age of Information (AoI), as a key indicator to measure the timeliness of data, is defined as the length of time from data generation to data reception by end users. In the dynamic renewable energy grid-connected environment, minimizing the age of information is the core of improving decision-making efficiency and optimizing energy dispatch response. Summary of the invention

[0006] The present invention aims to provide a strategy for minimizing the average information age of a drone-assisted network for new energy grid connection, so as to solve the above problems.

[0007] The technical solution of the present invention is: a new energy grid-connected drone-assisted network average information age minimization strategy, characterized by including the following:

[0008] S1: Establish relevant models for information age, including channel model, wireless transmission model, sensor node energy consumption model, drone energy consumption model, and drone charging model;

[0009] S2: Optimization modeling to minimize the average information age of sensor nodes;

[0010] S3: Construct an algorithm based on DQN to minimize the average information age, including:

[0011] S31: construct MDP, including constructing state space, action space, and reward function;

[0012] S32: The drone continuously learns through DQN to find the optimal action strategy, that is, for each state, finds the action that maximizes the weighted expected value of future rewards.

[0013] Preferably, in S1, the channel model and the wireless transmission model are specifically:

[0014] The channel gain between the two when the drone accesses the node is:

[0015]

[0016] In formula (1), d0 represents the path loss when the reference distance is 1m. is the path loss exponent, θ n represents the random phase of the wireless channel, which is assumed to be a random variable uniformly distributed in [0,2π];

[0017] The path loss is expressed as:

[0018]

[0019] Node I c The energy E that can be obtained in a single charging process Ic It is expressed as:

[0020]

[0021] In formula (3), Represents sensor node I c The wireless energy transmission time, η1 represents the conversion efficiency from RF to DC, and P1 is the RF transmission power of the drone;

[0022] Node I c The total amount of data that can be uploaded in the cth decision can be expressed as:

[0023]

[0024] In formula (4), Represents sensor node I c The wireless data upload time, P2 and B represent the sensor node signal transmission power and wireless channel bandwidth respectively, σ 2 is the additive white Gaussian noise (AWGN) power, Γ represents the difference between the theoretical value and the actual modulation and coding; sensor node I c The wireless energy transmission and data upload time can be expressed as:

[0025]

[0026] Preferably, in S1, the sensor node energy consumption model is specifically:

[0027] The monitoring energy consumption and data uploading energy consumption can be expressed as:

[0028]

[0029] In formula (7), P3 is the average power of the sensor node for environmental monitoring;

[0030] After the drone makes a decision, the remaining power of the sensor nodes that are not wirelessly powered during the current decision period It can be expressed as:

[0031]

[0032] in, Indicates the maximum power of each sensor node.

[0033] Preferably, in S1, the energy consumption model of the drone is specifically:

[0034] When the UAV is flying in a straight line at a uniform speed, the energy consumption model can be approximated as follows:

[0035] P(V)=c1(1+c2V 2 )

[0036]

[0037] In formula (10), c1, c2, c3, c4, c5, c6 are modeling parameters determined according to information such as fuselage resistance, air density, and rotor disk area; c1(1+c2V 2 ) represents the blade profile power, Indicates induced power, c6V 3 represents parasitic power;

[0038] When the UAV is executing the cth decision, the flight energy consumption of the UAV can be expressed as:

[0039]

[0040] matrix is the matrix that records the change of the remaining airborne energy of the UAV, E c UAC is the remaining airborne energy of the UAV after the UAV executes the cth decision. The calculation process is:

[0041]

[0042] Adoption Matrix It represents the energy required by the drone to go to each wireless charging platform after the drone executes the cth decision, which needs to meet the following constraints:

[0043] .

[0044] Preferably, in S1, the drone charging model is specifically:

[0045] According to Kirchhoff's law and the system resonant frequency f when the charging coil is in resonance, the system received power P is obtained by simplification. L And the power transmission efficiency η2 is:

[0046]

[0047] In formula (14), R 11 and R 22 are the equivalent resistances of the transmitting side and the receiving side respectively, ω is the system resonant angular frequency, and the mutual inductance

[0048] Preferably, S2 specifically includes:

[0049] Under the premise of avoiding the exhaustion of power of the UAV and each sensor node during the mission cycle, the average AoI of each sensor node is minimized by optimizing the UAV flight path planning, UAV charging strategy, and wireless power supply solution. The optimization problem can be modeled as:

[0050]

[0051] Preferably, in S31, constructing the state space S is specifically:

[0052] In the wireless power supply sensor network system of the wireless charging platform, the drone forms a state space by sensing the state information and dynamic changes of itself and each sensor node. This state space directly affects the drone's decision-making and system stability. Considering the time, location and state of each sensor node after the drone completes a decision, especially the remaining power of the drone and the sensor node, the information volume AoIA of each node c,k And the total amount of information in the drone decision cycle These factors jointly determine the action strategy of the drone, comprehensively consider the power and information volume of the drone and sensor nodes, and deduce the state of the drone after completing several decisions.

[0053] Preferably, in S31, constructing the action space A is specifically as follows:

[0054] According to I c The state divides decisions into four types:

[0055] Decision Type 1: When I c ≤K, and Status cThe updates are as follows:

[0056]

[0057] Decision Type 2: When I c ≤K, and s c The updates are as follows:

[0058]

[0059] In formula (28) l c The update is the same as (19) in decision type 1;

[0060] Decision Type 3: When I c ≤K, and s c The updates of each item can be expressed by equations (18), (21)-(24) and (26);

[0061] Decision Type 4: When I c >K, state s c The updates are as follows:

[0062]

[0063] Status c The other updates can be expressed using equations (18) and (22).

[0064] Preferably, in S31, constructing the reward function R is specifically:

[0065] The information age reward function of the drone after the cth decision is defined as:

[0066]

[0067] The remaining reward function of the drone’s onboard energy is:

[0068]

[0069] The remaining power reward function of each sensor node is:

[0070]

[0071] Taking the above factors into consideration, the following reward function is designed:

[0072] r c (s c ,a c )=w1r1 c +w2r2 c +w3r3 c (35)

[0073] In formula (34), r c (s c ,a c ) indicates that the reward function of the drone when executing the cth decision is a function of the state s at the cth decision. c and action a c function; w1, w2, w3 are the balance weights of the reward function of information age, remaining energy onboard the UAV and remaining power of each sensor node.

[0074] Preferably, in S32, for each state s c , find the action that maximizes the weighted expected value R of future rewards Specifically:

[0075] The reward weighted expected value R can be expressed as:

[0076]

[0077] In formula (36), γ represents the discount factor, and the size of γ reflects the importance of subsequent rewards to the long-term return of the current step;

[0078] Use π(a|s)=P(a=a c |s=s c ) to represent the action strategy selected by the drone, and use the state value function V π (s) is used to measure the quality of the action strategy π; V π (s) can be expressed as the expected value of the weighted reward that the agent can obtain by selecting actions according to the action strategy π starting from the current state:

[0079]

[0080] In formula (37), s′ represents the next state, P(s′,r|s,a) is the state transition function; through the state value function V π (s) can get the action-state value function Q π (s,a), that is, under a given strategy π, the value function of taking action a in state s is:

[0081]

[0082] The optimal action a is obtained by the above method * The state transfer function P is needed when Q is used. However, in this environment, the state transfer function P is unknown, so it is necessary to use an iterative method to π (s,a) is estimated, and the iterative formula can be expressed as:

[0083]

[0084] The update method used is to minimize the Bellman error function, as follows:

[0085] L(θ r )=(Y DQN -Q(s,a;θ r )) 2 (40)

[0086]

[0087] In formula (41), Y DQN The target network Q t Policy Network Q r is an estimate of r The update of can be obtained by using the gradient descent method on formula (40):

[0088]

[0089] In formula (42), α is the learning rate; the target network Q t (s,a;θ t ) in the parameter θ t It will not be updated automatically, but after every fixed step length l, let θ t =θ r .

[0090] The beneficial effects of the present invention are:

[0091] By deploying wireless charging platforms in the network, the charging round trips of drones are significantly reduced, effectively reducing the average AoI of the system and improving the operating efficiency. In addition, the trajectory and charging strategy of drones are optimized by using an algorithm based on a deep Q network (DQN), allowing flexible adjustment of task priorities based on real-time data and energy requirements.

[0092] Simulation results show that compared with traditional methods, this strategy reduces the average information age of the system by 12.7% to 70.9%, significantly enhancing the data freshness and response speed of the new energy grid-connected system. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 A schematic diagram of a drone-assisted wireless power supply sensor network system with an integrated wireless charging platform provided in an embodiment of the present invention;

[0094] Figure 2 A performance comparison chart of the DQN-based average AoI minimization algorithm provided in an embodiment of the present invention and four other algorithm solutions;

[0095] Figure 3AoI variation diagram of a drone's single decision to access a sensor node under two solutions provided in an embodiment of the present invention;

[0096] Figure 4 The average AoI graph of different nodes under each scheme when the number of sensor nodes K=10 provided in the embodiment of the present invention. DETAILED DESCRIPTION

[0097] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. The implementation methods of the present invention are not limited thereto.

[0098] Example 1

[0099] 1. System model:

[0100] like Figure 1 As shown, the system is deployed in renewable energy generation facilities such as wind power or photovoltaic power stations N Charging platforms and K Sensor nodes are strategically arranged to maximize coverage area and data collection efficiency. Drones play a key role in this system, responsible for regularly wirelessly powering sensors and collecting data, and monitoring facility operation status in real time.

[0101] The mission time of the drone is represented by T max , and its flying speed is v. The coordinate positions of the sensor node and the charging platform are and w k =[x k ,y k ,z k ] T , they are all located on the horizontal plane. The set of ground nodes is Μ={1,2,...,M},M=N+K, the matrix Used to represent the flight time of a drone from one ground node to another.

[0102] The drone determines its action strategy based on the remaining power AoI of the sensor node and the task sequence to optimize task execution and charging. c ∈Μ represents the location node number of the drone in each decision, Record the AoI of each sensor node and use Record the remaining power of the sensor node to further optimize the drone's decision-making.

[0103] 2. Problem Modeling

[0104] 2.1 Channel Model and Wireless Transmission Model

[0105] The channel gain between the two when the drone accesses the node is:

[0106]

[0107] In formula (1), d0 represents the path loss when the reference distance is 1m. is the path loss exponent, θ n represents the random phase of the wireless channel, which is assumed to be a random variable uniformly distributed in [0,2π].

[0108] The path loss is expressed as:

[0109]

[0110] Node I c The energy E that can be obtained in a single charging process Ic It is expressed as:

[0111]

[0112] In formula (3), Represents sensor node I c The wireless energy transmission time, η1 represents the conversion efficiency from RF to DC, and P1 is the RF transmission power of the drone.

[0113] Node I c The total amount of data that can be uploaded in the cth decision can be expressed as:

[0114]

[0115] In formula (4), Represents sensor node I c The wireless data upload time, P2 and B represent the sensor node signal transmission power and wireless channel bandwidth respectively, σ 2 is the additive white Gaussian noise (AWGN) power, and Γ represents the difference between the theoretical value and the actual modulation and coding. c The wireless energy transmission and data upload time can be expressed as:

[0116]

[0117] 2.2 Sensor Node Energy Consumption Model

[0118] The monitoring energy consumption and data uploading energy consumption can be expressed as:

[0119]

[0120] In formula (7), P3 is the average power of the sensor node for environmental monitoring.

[0121] After the drone makes a decision, the remaining power of the sensor nodes that are not wirelessly powered during the current decision period It can be expressed as:

[0122]

[0123] in, Indicates the maximum power of each sensor node.

[0124] 2.3 UAV energy consumption model

[0125] When the UAV is flying in a straight line at a uniform speed, the energy consumption model can be approximated as follows:

[0126] P(V)=c1(1+c2V 2 )

[0127]

[0128] In formula (10), c1, c2, c3, c4, c5, c6 are modeling parameters determined based on information such as fuselage resistance, air density, and rotor disk area. 2 ) represents the blade profile power, Indicates induced power, c6V 3 represents the parasitic power.

[0129] When the UAV is executing the cth decision, the flight energy consumption of the UAV can be expressed as:

[0130]

[0131] matrix is the matrix that records the change of the remaining airborne energy of the UAV, E c UAC is the remaining airborne energy of the UAV after the UAV executes the cth decision. The calculation process is:

[0132]

[0133] The present invention uses a matrix It represents the energy required by the drone to go to each wireless charging platform after the drone executes the cth decision, which needs to meet the following constraints:

[0134]

[0135] 2.4 Drone Charging Model

[0136] According to Kirchhoff's law and the system resonant frequency f when the charging coil is in resonance, the system received power P is obtained by simplification. L And the power transmission efficiency η2 is:

[0137]

[0138] In formula (14), R 11 and R 22 are the equivalent resistances of the transmitting side and the receiving side respectively, ω is the system resonant angular frequency, and the mutual inductance

[0139] 2.5 Problem Modeling

[0140] The goal of this invention is to minimize the average AoI of each sensor node by optimizing the UAV flight path planning, UAV charging strategy, and wireless power supply solution while avoiding the exhaustion of power of the UAV and each sensor node during the mission cycle. The optimization problem can be modeled as:

[0141]

[0142] 3. Average AoI minimization algorithm based on DQN

[0143] 3.1 Construction of MDP

[0144] MDP is a classic formal expression of sequential decision making and can be represented by a quintuple M = (S, A, P, R, π).

[0145] (1) State space S

[0146] In the wireless power supply sensor network system of the wireless charging platform, the drone forms a state space by sensing the state information and dynamic changes of itself and each sensor node, which directly affects the drone's decision-making and system stability. The present invention mainly considers the time, location and state of each sensor node after the drone completes a decision, especially the remaining power of the drone and the sensor node, the information volume AoIA of each node, and the time and location of the drone after completing a decision. c,k And the total amount of information in the drone decision cycle These factors together determine the action strategy of the drone. For example, when the drone is low on power or some sensor nodes are low on power, it will give priority to going to the charging platform or nodes with low power. Taking into account the power and information of the drone and sensor nodes, the state of the drone after completing several decisions can be deduced.

[0147] (2) Action Space A

[0148] According to I c The state divides decisions into four types:

[0149] Decision Type 1: When I c ≤K, and Status c The updates are as follows:

[0150]

[0151] Decision Type 2: When I c ≤K, and s c The updates are as follows:

[0152]

[0153]

[0154] In formula (28) l c The update is the same as (19) in decision type 1.

[0155] Decision Type 3: When I c ≤K, and s c The updates of each term can be expressed using equations (18), (21)-(24) and (26).

[0156] Decision Type 4: When I c >K, state s c The updates are as follows:

[0157]

[0158] Status c The other updates can be expressed using equations (18) and (22).

[0159] (3) Reward function R

[0160] The information age reward function of the drone after the cth decision is defined as:

[0161]

[0162] The remaining reward function of the drone’s onboard energy is:

[0163]

[0164] The remaining power reward function of each sensor node is:

[0165]

[0166] Taking the above factors into consideration, the following reward function is designed:

[0167]

[0168] In formula (34), r c (s c ,a c) indicates that the reward function of the drone when executing the cth decision is a function of the state s at the cth decision. c and action a c w1, w2, w3 are the balance weights of the reward function of information age, remaining energy onboard the drone and remaining power of each sensor node.

[0169] 3.2 Proposed Algorithm

[0170] The drone continuously learns through DQN to find the optimal action strategy, that is, for each state s c , find the action that maximizes the weighted expected value R of future rewards Among them, the reward weighted expected value R can be expressed as:

[0171]

[0172] In formula (36), γ represents the discount factor, and the size of γ reflects the importance of subsequent rewards to the long-term return of the current step.

[0173] Use π(a|s)=P(a=a c |s=s c ) to represent the action strategy selected by the agent (drone), and use the state value function V π (s) is used to measure the quality of the action strategy π. π (s) can be expressed as the expected value of the weighted reward that the agent can obtain by selecting actions according to the action strategy π starting from the current state:

[0174]

[0175] In formula (37), s′ represents the next state, and P(s′,r|s,a) is the state transition function. π (s) can get the action-state value function Q π (s,a), that is, under a given strategy π, the value function of taking action a in state s is:

[0176]

[0177] The optimal action a is obtained by the above method * The state transfer function P is needed when Q is used. However, in this environment, the state transfer function P is unknown, so it is necessary to use an iterative method to π (s,a) is estimated, and the iterative formula can be expressed as:

[0178]

[0179] The update method used is to minimize the Bellman error function, as follows:

[0180]

[0181] In formula (41), Y DQN The target network Q t Policy Network Q r Therefore, the parameter θ r The update of can be obtained by using the gradient descent method on formula (40):

[0182]

[0183] In formula (42), α is the learning rate. The target network Q t (s,a;θ t ) in the parameter θ t It will not be updated automatically, but after every fixed step length l, let θ t =θ r .

[0184] Table 1 Average AoI minimization algorithm based on DQN

[0185]

[0186]

[0187] The average AoI minimization algorithm based on DQN is shown in Table 2. In the algorithm, the target network Q is first initialized. t and policy network Q r The relevant parameter θ in t ,θ r , the drone randomly selects an action according to the current state s, and the decision process (s, a, r, s′) is stored in the experience replay pool D. When the experience pool is full, the drone will randomly extract N from the experience pool when making each decision. b Sample training is used to update the network parameters. After a fixed step length l, the policy network Q r The parameter θ in r Synchronize to target network Q t middle.

[0188] 4. Algorithm performance analysis

[0189] Figure 2 The performance comparison of the DQN-based minimization average AoI algorithm proposed in this invention with the other four algorithm solutions, where the horizontal axis is the training round (a flight cycle T max), the vertical axis is the average AoI of all nodes in the system. It can be seen that with the continuous increase in the number of training rounds, the average AoI of the algorithm proposed in the present invention is much lower than the first three comparison schemes, which fully demonstrates the superiority of the heuristic UAV path planning algorithm based on DQN in the simulation scenario of this section. At the same time, the performance of the algorithm proposed in the present invention is slightly better than the traditional UAV-assisted WPCN scheme. The average AoI of the two schemes are 17.8s and 20.4s respectively. This is because in the algorithm proposed in the present invention, the UAV will decide whether to wirelessly power the sensor node and the order of wireless power supply and data collection based on the remaining power to the sensor node, which can effectively reduce the AoI of the node.

[0190] Figure 3 The AoI changes of the drone’s single decision to access the sensor node under the two schemes. The blue line in the figure represents the remaining energy of the access sensor node higher than the dangerous power threshold E sensor,dangerous AoI changes when and They represent the remaining power of the access node in the cth decision-making process being higher or lower than the threshold E. sensor,enough end time. Compared with the traditional UAV-assisted WPCN scheme, the algorithm proposed in this invention collects data at an earlier time to reduce AoI when the access node has sufficient power. The AoI change curves of the two schemes are the same when and only when the remaining power of the sensor node is too low. The green shaded part in the figure shows the difference in AoI between the two schemes in a single decision, which is also the reason for the difference in average AoI between the two schemes.

[0191] Figure 4 is the average AoI of different nodes under each scheme when the number of sensor nodes K=10. It can be seen that compared with the other four algorithms, the algorithm proposed in this invention reduces the average AoI by 70.9%, 54.9%, 45.2% and 12.7%. Compared with the first three algorithms, the difference in the average AoI of each sensor node between the traditional UAV-assisted WPCN scheme and the algorithm proposed in this invention is small. This is because the same reward function is used in both schemes, which takes the maximum sensor node instantaneous information age as In this consideration, the problem of the average AoI of a certain sensor node being too large or too small is avoided, and more attention is paid to fairness.

[0192] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A strategy for minimizing the average information age of a drone-assisted network for renewable energy grid connection, characterized in that: These include: S1: Establish relevant models for information age, including channel model, wireless transmission model, sensor node energy consumption model, drone energy consumption model, and drone charging model; S2: Optimization modeling to minimize the average information age of sensor nodes; S3: Construct an algorithm based on DQN to minimize the average information age, including: S31: construct MDP, including constructing state space, action space, and reward function; S32: The drone continuously learns through DQN to find the optimal action strategy, that is, for each state, finds the action that maximizes the weighted expected value of future rewards.

2. The strategy for minimizing the average information age of a drone-assisted network for renewable energy grid connection according to claim 1 is characterized in that: In S1, the channel model and wireless transmission model are specifically as follows: The channel gain between the two when the drone accesses the node is: In formula (1), d0 represents the path loss when the reference distance is 1m. is the path loss exponent, θ n represents the random phase of the wireless channel, which is assumed to be a random variable uniformly distributed in [0,2π]; The path loss is expressed as: Node I c The energy E that can be obtained in a single charging process Ic It is expressed as: In formula (3), Represents sensor node I c The wireless energy transmission time, η1 represents the conversion efficiency from RF to DC, and P1 is the RF transmission power of the drone; Node I c The total amount of data that can be uploaded in the cth decision can be expressed as: In formula (4), Represents sensor node I c The wireless data upload time, P2 and B represent the sensor node signal transmission power and wireless channel bandwidth respectively, σ 2 is the additive white Gaussian noise (AWGN) power, Γ represents the difference between the theoretical value and the actual modulation and coding; sensor node I c The wireless energy transmission and data upload time can be expressed as:

3. The strategy for minimizing the average information age of a drone-assisted network for renewable energy grid connection according to claim 1 is characterized in that: In S1, the energy consumption model of the sensor node is as follows: The monitoring energy consumption and data uploading energy consumption can be expressed as: In formula (7), P3 is the average power of the sensor node for environmental monitoring; After the drone makes a decision, the remaining power of the sensor nodes that are not wirelessly powered during the current decision period It can be expressed as: in, Indicates the maximum power of each sensor node.

4. The strategy for minimizing the average information age of a drone-assisted network for renewable energy grid connection according to claim 1 is characterized in that: In S1, the energy consumption model of the drone is as follows: When the UAV is flying in a straight line at a uniform speed, the energy consumption model can be approximated as follows: In formula (10), c1, c2, c3, c4, c5, c6 are modeling parameters determined according to information such as fuselage resistance, air density, and rotor disk area; c1(1+c2V 2 ) represents the blade profile power, Indicates induced power, c6V 3 represents parasitic power; When the UAV is executing the cth decision, the flight energy consumption of the UAV can be expressed as: matrix A matrix that records the changes in the remaining onboard energy of the drone. is the remaining airborne energy of the UAV after the UAV executes the cth decision. The calculation process is: Adoption Matrix It represents the energy required by the drone to go to each wireless charging platform after the drone executes the cth decision, which needs to meet the following constraints:

5. The strategy for minimizing the average information age of a drone-assisted network for renewable energy grid connection according to claim 1 is characterized in that: In S1, the drone charging model is as follows: According to Kirchhoff's law and the system resonant frequency f when the charging coil is in resonance, the system received power P is obtained by simplification. L And the power transmission efficiency η2 is: In formula (14), R 11 and R 22 are the equivalent resistances of the transmitting side and the receiving side respectively, ω is the system resonant angular frequency, and the mutual inductance 6. The strategy for minimizing the average information age of a drone-assisted network for renewable energy grid connection according to claim 1 is characterized in that: S2 specifically includes: Under the premise of avoiding the exhaustion of power of the UAV and each sensor node during the mission cycle, the average AoI of each sensor node is minimized by optimizing the UAV flight path planning, UAV charging strategy, and wireless power supply solution. The optimization problem can be modeled as:

7. The strategy for minimizing the average information age of a drone-assisted network for renewable energy grid connection according to claim 1 is characterized in that: In S31, the construction state space S is specifically: In the wireless power supply sensor network system of the wireless charging platform, the drone forms a state space by sensing the state information and dynamic changes of itself and each sensor node. This state space directly affects the drone's decision-making and system stability. Considering the time, location and state of each sensor node after the drone completes a decision, especially the remaining power of the drone and the sensor node, the information volume AoIA of each node c,k And the total amount of information in the drone decision cycle These factors jointly determine the action strategy of the drone, comprehensively consider the power and information volume of the drone and sensor nodes, and deduce the state of the drone after completing several decisions.

8. The strategy for minimizing the average information age of a drone-assisted network for renewable energy grid connection according to claim 1 is characterized in that: In S31, the action space A is constructed as follows: According to I c The state divides decisions into four types: Decision Type 1: When I c ≤K, and Status c The updates are as follows: Decision Type 2: When I c ≤K, and s c The updates are as follows: In formula (28) l c The update is the same as (19) in decision type 1; Decision Type 3: When I c ≤K, and s c The updates of each item can be expressed by equations (18), (21)-(24) and (26); Decision Type 4: When I c >K, state s c The updates are as follows: Status c The other updates can be expressed using equations (18) and (22).

9. The strategy for minimizing the average information age of a drone-assisted network for renewable energy grid connection according to claim 1 is characterized in that: In S31, the reward function R is constructed as follows: The information age reward function of the drone after the cth decision is defined as: The remaining reward function of the drone’s onboard energy is: The remaining power reward function of each sensor node is: Taking the above factors into consideration, the following reward function is designed: r c (s c ,a c )=w1r1 c +w2r2 c +w3r3 c (35) In formula (34), r c (s c ,a c ) indicates that the reward function of the drone when executing the cth decision is a function of the state s at the cth decision. c and action a c function; w1, w2, w3 are the balance weights of the reward function of information age, remaining energy onboard the drone and remaining power of each sensor node.

10. The strategy for minimizing the average information age of a drone-assisted network for renewable energy grid connection according to claim 1 is characterized in that: In S32, for each state s c , find the action that maximizes the weighted expected value R of future rewards Specifically: The reward weighted expected value R can be expressed as: In formula (36), γ represents the discount factor, and the size of γ reflects the importance of subsequent rewards to the long-term return of the current step; Use π(a|s)=P(a=a c |s=s c ) to represent the action strategy selected by the drone, and use the state value function V π (s) is used to measure the quality of the action strategy π; V π (s) can be expressed as the expected value of the weighted reward that the agent can obtain by selecting actions according to the action strategy π starting from the current state: In formula (37), s′ represents the next state, P(s′,r|s,a) is the state transition function; through the state value function V π (s) can get the action-state value function Q π (s,a), that is, under a given strategy π, the value function of taking action a in state s is: The optimal action a is obtained by the above method * The state transfer function P is needed when Q is used. However, in this environment, the state transfer function P is unknown, so it is necessary to use an iterative method to π (s,a) is estimated, and the iterative formula can be expressed as: The update method used is to minimize the Bellman error function, as follows: L(θ r )=(Y DQN -Q(s,a;θ r )) 2 (40) In formula (41), Y DQN The target network Q t Policy Network Q r is an estimate of r The update of can be obtained by using the gradient descent method on formula (40): In formula (42), α is the learning rate; the target network Q t (s,a;θ t ) in the parameter θ t It will not be updated automatically, but after every fixed step length l, let θ t =θ r .

Images