Underwater wireless rechargeable sensor network charging scheduling method based on uwdqn

Abstract

The application discloses a kind of underwater wireless rechargeable sensor network charging scheduling methods based on UWDQN, it is related to wireless rechargeable sensor network field, including: the underwater wireless rechargeable sensor network model of including base station, underwater mobile charger and underwater sensor node is constructed;Underwater mobile charger motion energy consumption model is constructed;Underwater sensor node node energy consumption model is constructed;Underwater mobile charger charging energy consumption model is constructed;Based on UWDQN method, the charging node path of underwater mobile charger is determined.The application can effectively improve the charging efficiency of wireless rechargeable sensor network in underwater three-dimensional environment, overcome the adverse effects of sensor energy shortage on sensor network.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of wireless rechargeable sensor networks, and more particularly to a charging scheduling method for an underwater wireless rechargeable sensor network based on UWDQN. BACKGROUND

[0002] With the development of swarm intelligence, wireless rechargeable sensor networks have also attracted much attention. A wireless rechargeable sensor network is a self-organizing network structure composed of a plurality of sensor nodes, which collect, process and transmit environment-related information and deliver the information to a base station in a multi-hop or self-organizing manner, in the process of which energy is lost. The energy problem of sensors has always limited the development of wireless rechargeable sensor networks, so the design of charging strategies and charging scheduling methods is crucial.

[0003] However, most of the current research focuses on charging scheduling on the ground and in two-dimensional environments, and the research on charging scheduling in underwater environments is slightly blank. However, underwater sensor networks play a crucial role in both marine environment monitoring and even national defense and military fields. At the same time, the node energy in the underwater three-dimensional environment has a great influence on the charging scheduling scheme of the mobile charger in the wireless rechargeable sensor network, and most of the existing researches ignore the problem of dynamic energy consumption of nodes. Therefore, it is urgent to design a wireless rechargeable sensor network model in an underwater environment and to design a charging scheduling method, and to apply the dynamic energy consumption model to the three-dimensional underwater network environment to solve the problem of insufficient sensor energy in the underwater environment affecting the service life of the sensor network.

[0004] Therefore, how to schedule the underwater mobile charger is a problem that those skilled in the art need to solve. SUMMARY

[0005] Therefore, the present application provides a charging scheduling method for an underwater wireless rechargeable sensor network based on UWDQN, which solves the above problems.

[0006] In order to achieve the above purpose, the present application provides the following technical scheme:

[0007] The present application discloses a charging scheduling method for an underwater wireless rechargeable sensor network based on UWDQN, and the specific steps are as follows:

[0008] Step 1: constructing an underwater wireless rechargeable sensor network model comprising a base station, an underwater mobile charger and underwater sensor nodes;

[0009] Step 2: constructing a motion energy consumption model of the underwater mobile charger;

[0010] Step 3: constructing a node energy consumption model of the underwater sensor nodes;

[0011] Step 4: Constructing the charging energy consumption model of the underwater mobile charger;

[0012] Step 5: Determining the charging node path of the underwater mobile charger based on the UWDQN method.

[0013] Further, the underwater wireless rechargeable sensor network model comprises:

[0014] An underwater three-dimensional coordinate system is established to determine the positions of base stations, underwater mobile chargers, sensor nodes and the routing relationship between sensor nodes in the underwater wireless rechargeable sensor network.

[0015] The energy, charging energy threshold and moving energy threshold of the underwater mobile charger are determined.

[0016] The maximum battery capacity, charging request threshold and death threshold of each sensor node are determined.

[0017] Further, the formula of the motion energy consumption model is:

[0018] ;

[0019] wherein, represents the energy consumption of the underwater mobile charger from node to node , m represents the mass of the underwater mobile charger, g is the acceleration of gravity, is the density of water, V is the displacement volume of the underwater mobile charger, represents the vertical distance between node and node , represents the water resistance energy consumption.

[0020] Further, the formula of the node energy consumption model is:

[0021] ;

[0022] wherein, represents the data transmission energy consumption rate of node , represents the energy consumption of receiving 1 bit of data, represents the set of other nodes forwarding data through node , represents the data amount generated by node in , represents the data generated by node itself per second, Energy consumption per bit data from node Energy consumption for transmission to next hop.

[0023] Further, the charging energy consumption model comprises:

[0024] Energy consumption when the node is partially charged in mode A, if the remaining energy of the node is less than the charging request threshold, if , then the charging amount of the underwater mobile charger to the node is:

[0025] ;

[0026] wherein, represents the data transmission energy consumption rate of the node ; and respectively represent the maximum battery capacity and the charging request threshold of the node; T represents the charging time of the node; represents the energy consumed by the underwater mobile charger to fully charge or partially charge the node ; represents the energy consumed by the underwater mobile charger to partially charge the node ; represents the remaining energy of the node ; represents the energy loss rate of wireless charging;

[0027] Energy consumption when the node is fully charged in mode B, if and , it is determined that the remaining energy of the underwater mobile charger is sufficient to fully charge, then the charging amount of the underwater mobile charger to the node is:

[0028] ;

[0029] wherein, represents the energy consumed by the underwater mobile charger to fully charge the node ;

[0030] Energy consumption of the underwater mobile charger to the node when the remaining charging energy of the underwater mobile charger is insufficient in mode C is: ; wherein, represents the remaining charging energy of the underwater mobile charger.

[0031] Further, the step 5 comprises:

[0032] ​Step 5.1: Based on the underwater wireless rechargeable sensor network model, determine the charging path representation of the underwater mobile charger; based on the mobile charger motion energy consumption model and the underwater mobile charger charging energy consumption model, determine the objectives and constraints of the charging path;

[0033] Step 5.2: Based on the underwater wireless rechargeable sensor network model, set the state space and action space of the Q network, confirm the iterative update formula of the Q network parameters, set the reward function based on the objective and constraints, and establish the UWDQN network;

[0034] Step 5.3: Train the UWDQN network. After the training converges, schedule the underwater mobile charger to perform the charging task according to the optimal action predicted by the UWDQN network.

[0035] Furthermore, the charging path is described as follows: ;in, Indicates base station, This indicates the last sensor node to be charged;

[0036] The objective is to maximize charging efficiency and minimize the dead node rate, expressed as:

[0037] ; ;

[0038] The constraint is that the charging energy and driving energy consumption are less than the capacity of each battery, expressed as:

[0039] ; ;

[0040] in, Indicates charging efficiency. Indicates the death rate; Represents a node state, n The total number of nodes. A value of 1 indicates Death invalidation, 0 indicates Survival; and These represent the charging energy threshold and the mobile energy threshold of the underwater mobile charger, respectively. This indicates that charging consumes energy. , Indicates the energy consumed during driving. , and The underwater mobile charger is represented as a node. Energy consumed during charging and driving to the node The energy consumed.

[0041] Furthermore, the state space s This includes: the location, energy, and energy consumption of all nodes, as well as the location and energy of the underwater mobile charger; the action space. This includes: the location of all nodes and the location of the base station;

[0042] The iterative update formula is:

[0043] ;

[0044] in, These are the parameters of the current neural network. For learning rate, Represent the loss function as a whole Relative to parameters gradient, Let the mean squared error loss function be . This indicates the current state of the neural network. s Next action Value estimation; The target Q-value is calculated for the target network. For the parameters of the target network, Perform actions for instant rewards Then, immediate feedback signals are obtained from the environment; This is a discount factor used to balance the importance of current and future rewards; The target network is represented as a whole for the next state. Maximum Q-value estimation;

[0045] The reward function R The formula is:

[0046] ;

[0047] in, and All weights are greater than 0. and The sum is 1.

[0048] Furthermore, in the state space and the action space,

[0049] Underwater portable charger Energy in tense The formula is:

[0050] ;

[0051] in, Energy of the underwater mobile charger at q time t, Energy consumed by the underwater mobile charger moving from the last node to the next node ; Energy consumed by the underwater mobile charger charging the last node ;

[0052] Energy of the sensor node at time t is given by:

[0053] ;

[0054] wherein, Energy of the node at q time t , T is the sum of charging time and traveling time of the node, is the energy loss rate of wireless charging, is the charging efficiency of the underwater mobile charger charging the node, is the data transmission energy consumption rate of the node , and UMC represents the underwater mobile charger.

[0055] Further, the step 5.3 specifically comprises:

[0056] collecting information of the underwater mobile charger and information of the sensor node, updating the underwater wireless rechargeable sensor network model, and determining initial parameters of the state space and action space of the UWDQN network;

[0057] selecting an action through a strategy, randomly selecting a sensor node to be charged with a probability , and determining a charging amount according to the charging energy consumption model;

[0058] the agent selects an action with the largest current Q value with a probability ; the agent obtains a reward and updates parameters of the Q network, and stores contents of training samples of each step in an experience replay pool for sampling training;

[0059] randomly selecting a plurality of training samples from the experience replay pool, training and fitting the UWDQN network, and predicting an optimal action by using the converged UWDQN network;

[0060] scheduling the underwater mobile charger to charge each sensor node according to the optimal action sequence.

[0061] Compared with the prior art, the application discloses a charging scheduling method for an underwater wireless rechargeable sensor network based on a UWDQN, an underwater mobile charger energy consumption model, a dynamic energy consumption model of an underwater sensor, and a charging model of a mobile charger are constructed, and then a charging node is determined and the mobile charger is scheduled by taking the energy constraint of the mobile charger and the charging efficiency of the mobile charger as targets. BRIEF DESCRIPTION OF DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced as follows. Obviously, the drawings in the following description only constitute the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0063] Figure 1 It is a schematic diagram of the overall flow of the embodiment of the present application.

[0064] Figure 2 It is a schematic diagram of the underwater wireless rechargeable sensor network model of the embodiment of the present application.

[0065] Figure 3 It is a force diagram of the motion energy consumption model of the underwater mobile charger of the embodiment of the present application.

[0066] Figure 4 It is a schematic diagram of the sensor node information transmission path of the embodiment of the present application.

[0067] Figure 5 It is a schematic diagram of the information transmission path when there is a sensor node death of the embodiment of the present application.

[0068] Figure 6 It is a flowchart of the UWDQN method of the embodiment of the present application.

[0069] Figure 7 It is a routing schematic diagram of 50 nodes in a three-dimensional environment of the embodiment of the present application. DETAILED DESCRIPTION

[0070] Clearly, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of the present application.

[0071] The embodiment of the present application discloses a charging scheduling method for underwater wireless rechargeable sensor network based on UWDQN, as shown in Figure 1 The specific steps are as follows:

[0072] Step 1: Construct an underwater wireless rechargeable sensor network model containing a base station, an underwater mobile charger and underwater sensor nodes;

[0073] Step 2: Construct a motion energy consumption model of the underwater mobile charger;

[0074] Step 3: Construct a node energy consumption model of the underwater sensor node;

[0075] Step 4: Construct a charging energy consumption model of the underwater mobile charger;

[0076] Step 5: Based on the UWDQN method, determine the charging node path of the underwater mobile charger.

[0077] In one specific embodiment, the underwater wireless rechargeable sensor network model includes:

[0078] An underwater three-dimensional coordinate system is established to determine the positions of the base station, the underwater mobile charger and the sensor nodes in the underwater wireless rechargeable sensor network, as well as the routing relationship between the sensor nodes;

[0079] Determine the energy, charging energy threshold and moving energy threshold of the underwater mobile charger;

[0080] Determine the maximum battery capacity, charging request threshold and death threshold of each sensor node.

[0081] Specifically, the model layout of the underwater wireless rechargeable sensor network is as shown in Figure 2 The underwater wireless rechargeable sensor network is in a still water environment with a size of , wherein there is no other obstacle in the area, and there are n sensor nodes , nodes are randomly distributed in the region, each sensor node has a parent node, and the sensor forms a route with its parent node. Only one UMC charges for its one-to-one. SN obtains information from the surrounding environment and transmits data to BS in a multi-hop or self-organizing manner, and the energy consumption rate of SN is dynamically changed, and the death of the node associated with it will affect its energy consumption. The maximum battery capacity of SN , the SN sends a charging request to the BS when it reaches the charging request threshold , and the BS stores the charging request in the request sequence for subsequent operation, and when the energy of the SN is lower than the death threshold , it is considered dead. The base station BS is located at the center of the region, the initial position of the UMC is at the BS, the sensor nodes are randomly distributed in the region, and the position of the sensor node is fixed. The charging energy threshold of the UMC is , and the moving energy threshold is . When the battery energy of the UMC is insufficient to charge the SN or the mobile battery energy is about to be exhausted, the UMC returns to the BS. The distribution of 50 nodes in a three-dimensional environment is shown in Figure 7 .

[0082] In a specific embodiment, the underwater mobile charger is subjected to force as shown in Figure 3 , and the formula of the motion energy consumption model is:

[0083] ;

[0084] wherein, represents the energy consumption of the underwater mobile charger from node to node , m represents the mass of the underwater mobile charger, g is the acceleration of gravity, is the density of water, V is the displacement volume of the underwater mobile charger, represents the vertical distance between node and node , represents the energy consumption of water resistance, and the formula is:

[0085] ;

[0086] wherein, represents the drag coefficient of the fluid, v is the moving speed of the underwater mobile charger, represents the Euclidean distance between node and node , represents the water area, and the formula is:

[0087] ;

[0088] in, Indicates the semi-major axis of the ellipsoid. b Indicates the minor semi-axis of the ellipsoid. c Represents the minor semi-axis of the ellipsoid , q The angle of attack is expressed by the formula:

[0089] ;

[0090] in, and These represent the tilt angle and motion angle of the underwater mobile charger, respectively. To minimize the impact of the charging angle on charging efficiency, we define... Angle of motion with UMC The relationship formula is:

[0091] ;

[0092] .

[0093] When moving in still water, setting the direction of lift of this model perpendicular to the direction of motion will affect the drag coefficient. The magnitude of the drag coefficient at zero lift. The lift coefficient is Then the drag coefficient ,in, k It is a constant. , , .

[0094] In a specific embodiment, the energy consumption of the SN is mainly used for data transmission. Since the communication environment is underwater, the special characteristics of the underwater environment and their impact on underwater communication must be fully considered. Underwater acoustic communication is selected for modeling, and the formula for the node energy consumption model is as follows:

[0095] ;

[0096] in, Represents a node Data transmission energy consumption This represents the energy consumption for receiving 1 bit of data. Indicates passing through nodes The set of other nodes that forward data. express Middle node The amount of data generated Represents a node The data it generates per second, represents the energy consumption of each bit data from a node to the next hop, and the formula is:

[0097] ;

[0098] wherein, is a geometric transmission loss factor, and the value range in a three-dimensional underwater environment is [1, 2]; represents the Euclidean distance of a node to the next hop, and respectively represent the three-dimensional coordinates of the node and the next hop node, and the Euclidean distance is obtained according to the coordinates; represents the frequency f absorption coefficient, and the formula is:

[0099] .

[0100] The information transmission path of the sensor node in the application is shown in Figure 4 When all the nodes in the figure are alive, and the two points AB are within the communication range of each other, the nodes D, E and B are within the communication range, and the information transmission path is: , When the sensor node dies, the information transmission path in the application is shown in Figure 5 When node B dies, and nodes A, D and E are within the communication range of each other, the information transmission path is: , If when node B dies, the nodes D and E are not within the communication range of node A, the nodes D and E cannot communicate with node A, and are regarded as dead.

[0101] In a specific embodiment, the charging energy consumption model comprises:

[0102] The energy consumption when the node is charged in mode A, when the remaining power of the node is less than the charging request threshold, if , the charging amount of the underwater mobile charger to the node is:

[0103] ;

[0104] wherein, represents the data transmission energy consumption rate of the node ; and respectively represent the maximum battery capacity and the charging request threshold of the node; T represents the charging time of the node; represents the charging amount of the underwater mobile charger to the node energy consumed by full charging or partial charging; represents the remaining energy of the underwater mobile charger energy consumed by partial charging; represents the node the remaining energy of the underwater mobile charger; represents the energy loss rate of wireless charging;

[0105] energy consumption of mode B full charging, when and if the remaining energy of the underwater mobile charger is sufficient for full charging, the charging amount of the underwater mobile charger to the node is:

[0106] ;

[0107] wherein, represents the remaining energy of the underwater mobile charger energy consumed by full charging;

[0108] when the remaining charging energy of the underwater mobile charger is insufficient, i.e. , the charging amount of the underwater mobile charger to the node is: ; wherein, represents the remaining charging energy of the underwater mobile charger.

[0109] In one specific embodiment, step 5 comprises:

[0110] Step 5.1: based on the underwater wireless rechargeable sensor network model, determine the charging path expression of the underwater mobile charger; based on the mobile charger motion energy consumption model and the underwater mobile charger charging energy consumption model, determine the target and constraint of the charging path;

[0111] Step 5.2: according to the underwater wireless rechargeable sensor network model, set the state space and action space of the Q network, and confirm the iterative update formula of the Q network parameters, set the reward function based on the target and constraint, and establish the UWDQN network;

[0112] Step 5.3: as shown in Figure 6 , the UWDQN network is trained, and after the training converges, the optimal action predicted by the UWDQN network is used to schedule the underwater mobile charger to perform the charging task.

[0113] In one specific embodiment, the charging path expression is represented as: ; wherein, represents the base station, represents the last charged sensor node;

[0114] the target is to maximize the charging efficiency and minimize the dead node rate, which is represented as:

[0115] ; ;

[0116] The constraints are that the charging energy and the driving energy are less than the capacity of each battery, denoted as:

[0117] ; ;

[0118] wherein, denotes the charging efficiency, denotes the death rate of nodes; denotes the state of a node , n is the total number of nodes, is 1 indicating death failure, is 0 indicating survival; and denote the charging energy threshold and the moving energy threshold of the underwater mobile charger, respectively; denotes the charging consumption energy, , denotes the driving consumption energy , and denote the energy consumed by the underwater mobile charger to charge the node and the energy consumed by the underwater mobile charger to drive to the node .

[0119] In one specific embodiment, the state space s includes: the positions, energies, and energy consumptions of all nodes, and the positions and energy of the underwater mobile charger; the action space includes: the positions of all nodes and the position of the base station;

[0120] The iterative update formula is:

[0121] ;

[0122] wherein, is the parameter of the current neural network, is the learning rate, is an integral representing the gradient of the loss function with respect to the parameter , is the mean square error loss function, denotes the value estimate of the current neural network for the action s under the state ; a target Q-value computed for a target network target network, a parameter for the target network, performing an action for an immediate reward an immediate feedback signal obtained from the environment after; a discount factor balancing the importance of current and future rewards; a maximum Q-value estimate of the target network target network for a next state ;

[0123] the reward function R is formulated as:

[0124] ;

[0125] wherein, and are weights greater than 0, and add up to 1.

[0126] In particular, deep reinforcement learning is applied in the scheduling of underwater wireless rechargeable sensor network mobile chargers, and the UWDQN aims to make an agent learn how to perform in an environment based on the rewards it gets after performing certain actions. The base station as the agent first receives information from the environment, interacts with the environment, and then selects an action i.e. which sensor node to charge, from a set of available operations, and then interacts in the environment, after which the state of the environment is transformed from s to , the agent gets an immediate reward r , and the goal of the agent is to get as much reward as possible.

[0127] In a specific embodiment, in the state space and the action space,

[0128] the energy of the underwater mobile charger at temporal is formulated as:

[0129] ;

[0130] wherein, represents the energy of the underwater mobile charger at q temporal, represents the energy consumption of the underwater mobile charger moving from the last node to the next node ; represents the energy consumption of the underwater mobile charger charging the last node ;

[0131] ​sensor nodes exist Energy in tense The formula is:

[0132] ;

[0133] in, Represents a node exist q Temporal energy This represents the sum of the node's charging time and its travel time. T Charging time for nodes, This indicates the energy loss rate of wireless charging. This indicates the charging efficiency of the underwater mobile charger for charging the node. Represents a node The data transmission power consumption rate, UMC stands for Underwater Mobile Charger.

[0134] In one specific embodiment, step 5.3 specifically includes:

[0135] Collect information on underwater mobile chargers and sensor nodes, update the underwater wireless rechargeable sensor network model, and determine the initial parameters of the state space and action space of the UWDQN network.

[0136] pass Strategy selection action, in order to The probability is used to randomly select sensor nodes for charging, and the charging amount is determined according to the charging energy consumption model.

[0137] intelligent agents with Probability of choosing the current Q The action with the highest value; the agent receives the reward and updates. Q The network parameters will include the training sample content from each training step. done This is the termination flag; a value of 1 indicates the termination of a round, and the data is stored in the experience replay pool for sampling training.

[0138] A number of training samples are randomly selected from the experience replay pool, and the UWDQN network is used for training and fitting. The converged UWDQN network is then used to predict the optimal action. Specifically, a small batch of training samples is randomly selected from the experience replay pool and fed into the neural network for training and fitting. Q Networks are used for computing Q value ,in for Q Network parameters. The target network is used to calculate the target. Q value ,in The target network parameters are copied from the main network. Q The network has the same structure as the target network but different parameters. The loss function is then calculated The main network parameters are updated to minimize the loss function . Every step, the network is copied Q to the target network to update the target network. In the training phase above, the UMC does not perform real charging, only training with the agent, and all behaviors of the UMC are simulated in the UWDQN training of the BS.

[0139] Finally, the underwater mobile charger is scheduled to charge each sensor node according to the optimal action sequence. Specifically, when the training converges, the agent on the BS only selects a charging node according to the current state of the UMC and SN, directly queries the optimal action predicted by the UWDQN, and the UWDQN no longer performs gradient updates, stores new experience data, or explores. Based on the input state s directly outputs the optimal action . When the MC has an impact on the environment, repeat the above UWDQN algorithm operation. When the energy of the SN reaches the charging threshold, send a charging request to the base station, and at the same time send the state of the SN at this time, including the remaining power, energy consumption, and SN location. The base station selects the sensor node with the maximum reward for charging according to the charging request and the results of the training after convergence. Each time the task of the next charging node is transmitted to the UMC for execution and then acts on the environment, and then the agent repeats the above sampling training process.

[0140] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0141] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A charging scheduling method for underwater wireless rechargeable sensor networks based on UWDQN, characterized in that, The specific steps are as follows: Step 1: constructing an underwater wireless rechargeable sensor network model comprising a base station, an underwater mobile charger and underwater sensor nodes; Step 2: constructing a motion energy consumption model of the underwater mobile charger, the formula of which is: ； wherein represents the energy consumption of the underwater mobile charger from node to node , m represents the mass of the underwater mobile charger, g is the acceleration of gravity, is the density of water, V is the displacement volume of the underwater mobile charger, represents the vertical distance between node and node , represents the drag energy consumption of water; Step 3: constructing a node energy consumption model of the underwater sensor node, the formula of which is: ； wherein, denotes the energy consumption rate of a node , denotes the energy consumption of receiving 1 bit of data, denotes the set of other nodes through which a node forwards data, denotes the amount of data generated by a node , denotes the amount of data generated per second by a node , denotes the energy consumption per bit of data transmitted from a node to the next hop; Step 4: constructing a charging energy consumption model of the underwater mobile charger; Step 5: determining a charging node path of the underwater mobile charger based on the UWDQN method, comprising: Step 5.1: determining a charging path expression of the underwater mobile charger based on the underwater wireless rechargeable sensor network model; determining a target and a constraint of the charging path based on the motion energy consumption model of the mobile charger and the charging energy consumption model of the underwater mobile charger; Step 5.2: setting a state space and an action space of the Q network according to the underwater wireless rechargeable sensor network model, and confirming an iterative update formula of the Q network parameters; setting a reward function based on the target and the constraint, and establishing a UWDQN network; Step 5.3: training the UWDQN network, and scheduling the underwater mobile charger to perform a charging task according to an optimal action predicted by the UWDQN network after training convergence. 2.The UWDQN-based underwater wireless rechargeable sensor network charging scheduling method of claim 1, wherein, The underwater wireless rechargeable sensor network model comprises: establishing an underwater three-dimensional coordinate system to determine the positions of the base station, the underwater mobile charger and the sensor nodes in the underwater wireless rechargeable sensor network, and the routing relationship between the sensor nodes; determining the energy, charging energy threshold and moving energy threshold of the underwater mobile charger; determining the maximum battery capacity, charging request threshold and death threshold of each sensor node. 3.The UWDQN-based underwater wireless rechargeable sensor network charging scheduling method of claim 1, wherein, The charging energy consumption model comprises: Energy consumption of mode A part charging, when the remaining power of the node is less than the charging request threshold, if the charging amount of the underwater mobile charger to the node is: ； wherein, represents the data transmission energy consumption rate of a node ; and represent the maximum battery capacity and the charging request threshold of a node, respectively; T represents the charging time of a node; represents the energy consumed by the underwater mobile charger to fully charge or partially charge a node ; represents the energy consumed by the underwater mobile charger to partially charge a node ; represents the remaining energy of a node ; represents the energy loss rate of wireless charging; Energy consumption of mode B when fully charging, when and If the remaining energy of the underwater mobile charger is determined to be sufficient for full charging when , the charging amount of the underwater mobile charger to the node is: ； wherein, indicates that the underwater mobile charger is a node energy consumed for full charging; When the mode C underwater mobile charger is insufficient in residual charging energy, the charging amount of the underwater mobile charger to the node is: ; wherein, represents the residual charging energy of the underwater mobile charger. 4.The UWDQN-based underwater wireless rechargeable sensor network charging scheduling method of claim 1, wherein, The charging path expression is represented as: ; wherein, denotes a base station, denotes the last sensor node being charged; The target is to maximize the charging efficiency and minimize the death node rate, which is expressed as: ； ； The constraint is that the charging energy and driving energy are less than the battery capacity, which is expressed as: ； ； wherein, represents the charging efficiency, represents the death node rate; represents the state of the node , n is the total number of nodes, is 1 indicates death failure, is 0 indicates survival; and respectively represent the charging energy threshold and the moving energy threshold of the underwater mobile charger; represents the charging consumed energy, , represents the driving consumed energy , and represent the energy consumed by the underwater mobile charger to charge the node and the energy consumed to drive to the node .

5. The underwater wireless rechargeable sensor network charging scheduling method based on UWDQN of claim 4, wherein, The state space s includes: positions of all nodes, energy, energy consumption, and positions of underwater mobile chargers, energy; The action space includes: positions of all nodes and positions of base stations; The iterative update formula is: ； wherein, are parameters of the current neural network, is a learning rate, is a one-hot representation of the loss function with respect to the parameters of the neural network, is a mean squared error loss function, denotes the value estimate of the current neural network for the action s in state ; is the target Q-value computed by the target network, are parameters of the target network, is the immediate reward obtained from the environment after performing the action ; is a discount factor balancing the importance of current and future rewards; is a one-hot representation of the maximum Q-value estimate of the target network for the next state ; The reward function R The formula is: ； wherein, and are weights greater than 0, and add up to 1.

6. The underwater wireless rechargeable sensor network charging scheduling method based on UWDQN of claim 5, wherein, In the state space and the action space, An underwater mobile charger in Energy at a time The formula is: ； wherein, represents the energy of the underwater mobile charger at q temporal time, represents the energy consumed by the underwater mobile charger to move from the previous node to the next node ; represents the energy consumed by the underwater mobile charger to charge the previous node ; Sensor node In Energy at time The formula is: ； wherein, representing a node in q temporal energy, representing the sum of the charging time and the traveling time of a node, T is the charging time of a node, representing the energy loss rate of wireless charging, representing the charging efficiency of a node charged by an underwater mobile charger, representing a node data transmission energy consumption rate, UMC represents an underwater mobile charger.

7. The underwater wireless rechargeable sensor network charging scheduling method based on UWDQN of claim 1, wherein, The step 5.3 specifically comprises: collecting information of the underwater mobile charger and sensor nodes, updating the underwater wireless rechargeable sensor network model, and determining initial parameters of the state space and the action space of the UWDQN network; By selecting an action of a policy, and randomly selecting a sensor node to charge with a probability, and determining a charging amount according to the charging energy consumption model; The agent selects an action with the highest current Q value with a probability of The agent obtains a reward and updates the parameters of the Q network, and stores the training sample content of each step training sampling in an experience replay pool for sampling training. randomly selecting a plurality of training samples from the experience replay pool, training and fitting the UWDQN network, and predicting an optimal action by using the converged UWDQN network; scheduling the underwater mobile charger to charge each sensor node in the optimal action order.

Images