Underwater wireless rechargeable sensor network charging scheduling method based on UWDQN

Abstract

The invention discloses a UWDQN-based underwater wireless rechargeable sensor network charging scheduling method, and relates to the field of wireless rechargeable sensor networks, and the method comprises the steps: constructing an underwater wireless rechargeable sensor network model comprising a base station, an underwater mobile charger and underwater sensor nodes; constructing a motion energy consumption model of the underwater mobile charger; constructing a node energy consumption model of the underwater sensor nodes; constructing a charging energy consumption model of the underwater mobile charger; and determining a charging node path of the underwater mobile charger based on a UWDQN method. According to the invention, the charging efficiency of the wireless rechargeable sensor network in the underwater three-dimensional environment can be effectively improved, and the adverse effect of insufficient sensor energy on the sensor network is overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of wireless rechargeable sensor networks, and more particularly to a charging scheduling method for an underwater wireless rechargeable sensor network based on UWDQN. Background Art

[0002] With the development of intelligent robotics, wireless rechargeable sensor networks (WRNs) have also attracted significant attention. WRNs are self-organizing networks composed of multiple sensor nodes. These nodes collect, process, and transmit environmental information to a base station in a multi-hop or self-organizing manner, consuming energy in the process. Energy consumption has long been a limiting factor in the development of WRNs, making the design of charging strategies and scheduling methods crucial.

[0003] However, most current research focuses on charging scheduling on land and in two-dimensional environments, leaving relatively little research on charging scheduling in underwater environments. Underwater sensor networks, however, play a crucial role in both marine environmental monitoring and defense and military applications. Furthermore, node energy consumption in three-dimensional underwater environments significantly impacts the charging scheduling of mobile chargers in wireless rechargeable sensor networks. Existing research has largely ignored the issue of dynamic node energy consumption. Therefore, it is imperative to design a model for wireless rechargeable sensor networks in underwater environments and design a charging scheduling method for them. Applying dynamic energy consumption models to three-dimensional underwater network environments can address the issue of insufficient sensor energy in underwater environments, which impacts the lifespan of sensor networks.

[0004] Therefore, how to schedule underwater mobile chargers is a problem that those skilled in the art urgently need to solve. Summary of the Invention

[0005] In view of this, the present invention provides a charging scheduling method for an underwater wireless rechargeable sensor network based on UWDQN, which solves the above problems.

[0006] In order to achieve the above object, the present invention provides the following technical solutions: The present invention discloses a charging scheduling method for an underwater wireless rechargeable sensor network based on UWDQN, and the specific steps are as follows: Step 1: Construct an underwater wireless rechargeable sensor network model including a base station, an underwater mobile charger, and underwater sensor nodes; Step 2: Construct a motion energy consumption model for the underwater mobile charger; Step 3: Construct the node energy consumption model of underwater sensor nodes; Step 4: Construct a charging energy consumption model for the underwater mobile charger; Step 5: Based on the UWDQN method, determine the charging node path of the underwater mobile charger.

[0007] Furthermore, the underwater wireless rechargeable sensor network model includes: Establish an underwater three-dimensional coordinate system to determine the location of the base station, the underwater mobile charger, the location of each sensor node, and the routing relationship between each sensor node in the underwater wireless rechargeable sensor network; Determine the energy, charging energy threshold, and movement energy threshold of the underwater mobile charger; Determine the maximum battery capacity, charge request threshold, and death threshold of each sensor node.

[0008] Furthermore, the formula of the exercise energy consumption model is: ; in, Represents the underwater mobile charger slave node To Node energy consumption, m represents the mass of the underwater mobile charger, g is the acceleration due to gravity, is the density of water, V is the displacement volume of the underwater mobile charger, Representation node and nodes The vertical distance between Indicates the energy consumption of water resistance.

[0009] Furthermore, the formula of the node energy consumption model is: ; in, Representation node The data transmission energy consumption rate, Indicates the energy consumption of receiving 1 bit of data, Indicates that the node The set of other nodes that forward data, express midpoint The amount of data generated, Representation node The data generated per second, Indicates that each bit of data is from the node Energy consumption of transmission to the next hop.

[0010] Furthermore, the charging energy consumption model includes: Energy consumption during partial charging in mode A, when the node If the remaining power is less than the charge request threshold, , then the amount of charge that the underwater mobile charger provides to the node is: ; in, Representation node The energy consumption rate of data transmission; and They represent the maximum battery capacity and charging request threshold of the node respectively; T Indicates the node charging time; Indicates that the underwater mobile charger is a node The energy consumed for a full or partial charge; Indicates that the underwater mobile charger is a node Energy consumed by partial charging; Representation node The remaining energy; Indicates the energy loss rate of wireless charging; Energy consumption when fully charged in mode B, when and When it is determined that the remaining energy of the underwater mobile charger is sufficient to fully charge, the amount of charge of the node by the underwater mobile charger is: ; in, Indicates that the underwater mobile charger is a node Energy consumed for a full charge; In mode C, when the remaining charging energy of the underwater mobile charger is insufficient, the amount of charging the node by the underwater mobile charger is: ;in, Indicates the remaining charging energy of the underwater mobile charger.

[0011] Furthermore, the step 5 includes: Step 5.1: Determine a charging path representation of the underwater mobile charger based on the underwater wireless rechargeable sensor network model; determine the objectives and constraints of the charging path based on the mobile charger motion energy consumption model and the underwater mobile charger charging energy consumption model; Step 5.2: According to the underwater wireless rechargeable sensor network model, set the state space and action space of the Q network, confirm the iterative update formula of the Q network parameters, set the reward function based on the objectives and constraints, and establish the UWDQN network; Step 5.3: Train the UWDQN network. After the training converges, schedule the underwater mobile charger to perform the charging task according to the optimal action predicted by the UWDQN network.

[0012] Furthermore, the charging path is expressed as: ;in, represents a base station, Indicates the last sensor node to be charged; The goal is to maximize the charging efficiency and minimize the death node rate, which can be expressed as: ; ; The constraint is that the charging energy and driving energy consumption are less than the capacity of each battery, which can be expressed as: ; ; in, Indicates the charging efficiency, represents the death node rate; Representation node status, n is the total number of nodes, 1 means Death failure, 0 means Survival; and They represent the charging energy threshold and moving energy threshold of the underwater mobile charger respectively; Indicates the energy consumed by charging. , Indicates driving energy consumption , and Indicates that the underwater mobile charger is a node The energy consumed by charging and driving to the node The energy consumed.

[0013] Furthermore, the state space s Including: the location, energy, energy consumption of all nodes, and the location and energy of underwater mobile chargers; the action space Including: the location of all nodes and base stations; The iterative update formula is: ; in, are the parameters of the current neural network, is the learning rate, Represent the loss function as a whole Relative to the parameter The gradient, is the mean square error loss function, Indicates that the current neural network is in state s Next action Estimated value of The target Q value calculated for the target network target network, are the parameters of the target network, Performing actions for immediate rewards The immediate feedback signal obtained from the environment afterwards; is a discount factor used to balance the importance of current rewards and future rewards; As a whole, it represents the target network target network for the next state Maximum Q value estimation; The reward function R The formula is: ; in, and All weights are greater than 0. and Add up to 1.

[0014] Furthermore, in the state space and the action space, Underwater mobile charger Energy in tense The formula is: ; in, Indicates that the underwater mobile charger is q Energy in tense, Indicates that the underwater mobile charger has been transferred from the previous node Move to the next node Movement energy consumption; Indicates that the underwater mobile charger is given to the previous node Charging energy consumption; sensor nodes exist Energy in tense The formula is: ; in, Representation node exist q Tense energy, represents the sum of node charging time and travel time, T Charging time for the node, Indicates the energy loss rate of wireless charging, represents the charging efficiency of the underwater mobile charger to charge the node, Representation node The data transmission energy consumption rate, UMC stands for underwater mobile charger.

[0015] Furthermore, the step 5.3 specifically includes: Collect underwater mobile charger information and sensor node information, update the underwater wireless rechargeable sensor network model, and determine the initial parameters of the state space and action space of the UWDQN network; pass The strategy selects actions to The probability of randomly selecting a sensor node to be charged and determining the charging amount according to the charging energy consumption model; Intelligent agent The probability of selecting the action with the largest current Q value; the agent obtains the reward and updates the parameters of the Q network, and stores the training sample content of each step of training in the experience replay pool for sampling training; Randomly select a number of training samples from the experience replay pool, perform training and fitting on the UWDQN network, and use the converged UWDQN network to predict the optimal action; The underwater mobile charger is scheduled to charge each sensor node according to the optimal action sequence.

[0016] The above technical solution demonstrates that, compared to existing technologies, the present invention provides a charging scheduling method for underwater wireless rechargeable sensor networks based on UWDQN. This method constructs an underwater mobile charger energy consumption model, a dynamic energy consumption model for underwater sensors, and a charging model for the mobile charger. The method then uses mobile charger energy constraints and the goal of improving mobile charger charging efficiency to identify charging nodes and schedule the mobile chargers. This method can effectively improve the charging efficiency of wireless rechargeable sensor networks in underwater three-dimensional environments, overcoming the adverse effects of insufficient sensor energy on sensor networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0018] Figure 1 Schematic diagram of the overall process of an embodiment of the present invention; Figure 2 Schematic diagram of an underwater wireless rechargeable sensor network model according to an embodiment of the present invention; Figure 3 This is a force diagram of the motion energy consumption model of the underwater mobile charger according to an embodiment of the present invention; Figure 4 A schematic diagram of a sensor node information transmission path according to an embodiment of the present invention; Figure 5A schematic diagram of an information transmission path when a sensor node dies according to an embodiment of the present invention; Figure 6 Flowchart of the UWDQN method according to an embodiment of the present invention; Figure 7 Schematic diagram of routing of 50 nodes in a three-dimensional environment according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0020] The embodiment of the present invention discloses a charging scheduling method for an underwater wireless rechargeable sensor network based on UWDQN. Figure 1 The specific steps are as follows: Step 1: Construct an underwater wireless rechargeable sensor network model including a base station, an underwater mobile charger, and underwater sensor nodes; Step 2: Construct a motion energy consumption model for the underwater mobile charger; Step 3: Construct the node energy consumption model of underwater sensor nodes; Step 4: Construct a charging energy consumption model for the underwater mobile charger; Step 5: Based on the UWDQN method, determine the charging node path of the underwater mobile charger.

[0021] In a specific embodiment, the underwater wireless rechargeable sensor network model includes: Establish an underwater three-dimensional coordinate system to determine the location of the base station, the underwater mobile charger, the location of each sensor node, and the routing relationship between each sensor node in the underwater wireless rechargeable sensor network; Determine the energy, charging energy threshold, and movement energy threshold of the underwater mobile charger; Determine the maximum battery capacity, charge request threshold, and death threshold of each sensor node.

[0022] Specifically, the model layout of the underwater wireless rechargeable sensor network is as follows: Figure 2 As shown, underwater wireless rechargeable sensor network: in a In a still water environment with no other obstacles, there are n sensor nodes Nodes are randomly distributed in the area. Each sensor node has a parent node, and a route is formed between the sensor and its parent node. Only one UMC charges it one-to-one. SN obtains information from the surrounding environment and transmits data to the BS in a multi-hop or self-organizing manner. The energy consumption rate of SN changes dynamically. The death of the node it is connected to will affect its energy consumption. The maximum battery capacity of SN is , SN reaches the charging request threshold When the energy of SN is lower than the death threshold, BS sends a charging request to BS, which stores the charging request in the request sequence for subsequent operations. The base station BS is located at the center of the area, the initial position of the UMC is at the BS, the sensor nodes are randomly distributed in the area, and the positions of the sensor nodes are fixed. The charging energy threshold of the UMC is , the moving energy threshold is When the UMC battery energy is insufficient to charge the SN or the mobile battery energy is about to run out, the UMC returns to the BS. The 50 nodes in the three-dimensional environment are distributed as follows Figure 7 shown.

[0023] In a specific embodiment, the underwater mobile charger is subjected to a force such as Figure 3 As shown in Figure 2, the formula for the exercise energy consumption model is: ; in, Represents the underwater mobile charger slave node To Node energy consumption, m represents the mass of the underwater mobile charger, g is the acceleration due to gravity, is the density of water, V is the displacement volume of the underwater mobile charger, Representation node and nodes The vertical distance between The resistance energy consumption of water is expressed as follows: ; in, represents the resistance coefficient of the fluid, v is the moving speed of the underwater mobile charger, Representation node and nodes The Euclidean distance between represents the frontal area, and the formula is: ; in, represents the semi-major axis of the ellipsoid, brepresents the minor semi-axis of the ellipsoid, c Represents the minor semi-axis of the ellipsoid , q represents the angle of attack, and the formula is: ; in, and Represent the tilt angle and movement angle of the underwater mobile charger respectively. In order to minimize the impact of the charging angle on the charging efficiency, we define Movement angle with UMC The relationship formula is: ; .

[0024] When moving in a still water environment, the direction of the model's lift is set perpendicular to the direction of movement, which will affect the drag coefficient. The drag coefficient at zero lift is , the lift coefficient is , then the resistance coefficient ,in, k is a constant, , , .

[0025] In a specific embodiment, the energy consumption of SN is mainly used for data transmission. The communication environment is underwater. Therefore, the influence of the particularity of the underwater environment on underwater communication needs to be fully considered. The underwater acoustic communication method is selected for modeling. The formula of the node energy consumption model is: ; in, Representation node The data transmission energy consumption rate, Indicates the energy consumption of receiving 1 bit of data, Indicates that the node The set of other nodes that forward data, express midpoint The amount of data generated, Representation node The data generated per second, Indicates that each bit of data is from the node The energy consumption of transmission to the next hop is: ; in, is the geometric transmission loss factor, and its value range in the three-dimensional underwater environment is [1,2]; Representation node The Euclidean distance to the next hop, and Represents nodes respectively and the three-dimensional coordinates of the next hop node, and calculate the Euclidean distance based on the coordinates; Indicates frequency f The lower absorption coefficient is: .

[0026] The information transmission path of the sensor node in the present invention is as follows: Figure 4 As shown in the figure, when all nodes in the graph are alive, and points A and B are within each other's communication range, and nodes D, E, and B are within the communication range, the information transmission path is: 、 In the present invention, when a sensor node dies, the information transmission path is as follows: Figure 5 As shown in the figure, when node B dies and nodes A, D, and E are within each other's communication range, the information transmission path is: 、 If node B dies and nodes D and E are not within the communication range of node A, nodes D and E cannot communicate with node A and are considered dead.

[0027] In a specific embodiment, the charging energy consumption model includes: Energy consumption during partial charging in mode A, when the node If the remaining power is less than the charge request threshold, , then the amount of charge that the underwater mobile charger provides to the node is: ; in, Representation node The energy consumption rate of data transmission; and They represent the maximum battery capacity and charging request threshold of the node respectively; T Indicates the node charging time; Indicates that the underwater mobile charger is a node The energy consumed for a full or partial charge; Indicates that the underwater mobile charger is a node Energy consumed by partial charging; Representation node The remaining energy; Indicates the energy loss rate of wireless charging; Energy consumption when fully charged in mode B, when and When it is determined that the remaining energy of the underwater mobile charger is sufficient to fully charge, the amount of charge of the node by the underwater mobile charger is: ; in, Indicates that the underwater mobile charger is a node Energy consumed for a full charge; When the remaining charging energy of the underwater mobile charger in mode C is insufficient, , the amount of charge that the underwater mobile charger provides to the node is: ;in, Indicates the remaining charging energy of the underwater mobile charger.

[0028] In a specific embodiment, step 5 includes: Step 5.1: Determine the charging path representation of the underwater mobile charger based on the underwater wireless rechargeable sensor network model; determine the objectives and constraints of the charging path based on the mobile charger motion energy consumption model and the underwater mobile charger charging energy consumption model; Step 5.2: Based on the underwater wireless rechargeable sensor network model, set the state space and action space of the Q network, confirm the iterative update formula of the Q network parameters, set the reward function based on the objectives and constraints, and establish the UWDQN network; Step 5.3: If Figure 6 As shown in the figure, the UWDQN network is trained. After the training converges, the underwater mobile charger is scheduled to perform the charging task according to the optimal action predicted by the UWDQN network.

[0029] In a specific embodiment, the charging path is expressed as: ;in, represents a base station, Indicates the last sensor node to be charged; The goal is to maximize the charging efficiency and minimize the death node rate, which can be expressed as: ; ; The constraint is that the charging energy and driving energy consumption are less than the capacity of each battery, which can be expressed as: ; ; in, Indicates the charging efficiency, represents the death node rate; Representation node status, n is the total number of nodes, 1 means Death failure, 0 means Survival; and They represent the charging energy threshold and moving energy threshold of the underwater mobile charger respectively; Indicates the energy consumed by charging. , Indicates driving energy consumption , and Indicates that the underwater mobile charger is a node The energy consumed by charging and driving to the node The energy consumed.

[0030] In a specific embodiment, the state space s Including: the location, energy, energy consumption of all nodes, and the location and energy of underwater mobile chargers; the action space Including: the location of all nodes and base stations; The iterative update formula is: ; in, are the parameters of the current neural network, is the learning rate, Represent the loss function as a whole Relative to the parameter The gradient, is the mean square error loss function, Indicates that the current neural network is in state s Next action Estimated value of The target Q value calculated for the target network target network, are the parameters of the target network, Performing actions for immediate rewards The immediate feedback signal obtained from the environment afterwards; is a discount factor used to balance the importance of current rewards and future rewards; As a whole, it represents the target network target network for the next state Maximum Q value estimation; The reward function R The formula is: ; in, and All weights are greater than 0. and Add up to 1.

[0031] Specifically, deep reinforcement learning is applied to the scheduling of mobile chargers in underwater wireless rechargeable sensor networks. UWDQN aims to enable the agent to learn how to behave in the environment based on the rewards obtained after performing certain actions. The base station, as an agent, first receives environmental information and interacts with the environment. Then it selects an action from a set of available actions. That is, which sensor node is charged and then interacts in the environment, and then the environment state changes from s Convert to , the agent receives immediate rewards r , the goal of the agent is to get as much reward as possible.

[0032] In a specific embodiment, in the state space and action space, Underwater mobile charger Energy in tense The formula is: ; in, Indicates that the underwater mobile charger is q Energy in tense, Indicates that the underwater mobile charger has been transferred from the previous node Move to the next node Movement energy consumption; Indicates that the underwater mobile charger is given to the previous node Charging energy consumption; sensor nodes exist Energy in tense The formula is: ; in, Representation node exist q Tense energy, represents the sum of node charging time and travel time, T Charging time for the node, Indicates the energy loss rate of wireless charging, represents the charging efficiency of the underwater mobile charger to charge the node, Representation node The data transmission energy consumption rate, UMC stands for underwater mobile charger.

[0033] In a specific embodiment, step 5.3 specifically includes: Collect underwater mobile charger information and sensor node information, update the underwater wireless rechargeable sensor network model, and determine the initial parameters of the state space and action space of the UWDQN network; pass The strategy selects actions to The sensor nodes to be charged are randomly selected with a probability of , and the charging amount is determined according to the charging energy consumption model; Intelligent agent The probability of choosing the current Q The action with the largest value; the agent gets the reward and updates Q The parameters of the network, the content of the training samples sampled in each step of training done It is the abort flag. When it is 1, a round of abort is stored in the experience replay pool for sampling training. Randomly select a number of training samples from the experience replay pool, and the UWDQN network performs training fitting. The converged UWDQN network is used to predict the optimal action. Specifically, a small batch of training samples is randomly selected from the experience replay pool and input into the neural network for training fitting. The main network Q Networks for computing Q value ,in for Q Network parameters. The target network is used to calculate the target Q value ,in It is the target network parameter. Q The network has the same structure as the target network, but the parameters are different. Then the loss function is calculated. , update the main network parameters with the goal of minimizing the loss function .Every Step Copy Q The network is given to the target network to update the target network. In the above training phase, the UMC does not perform real charging and is only trained with the agent. All the behaviors of the UMC are simulated in the UWDQN training of the BS.

[0034] Finally, the underwater mobile charger is scheduled to charge each sensor node in the optimal action sequence. Specifically, after the training converges, the intelligent agent on the BS only queries the optimal action predicted by the UWDQN when selecting a charging node based on the current state of the UMC and SN. The UWDQN no longer performs gradient updates, stores new experience data, or explores, but only selects the optimal action based on the input state. s Directly output the optimal action . When MC has an impact on the environment, the above UWDQN algorithm operation is repeated. When the energy of SN reaches the charging threshold, it sends a charging request to the base station and simultaneously sends its current status including remaining power, energy consumption, and SN location. The base station selects the sensor node with the maximum reward to charge based on the charging request and the results after training convergence. Each time the next node task of charging is transmitted to UMC for execution and then acts on the environment, the agent repeats the above sampling training process.

[0035] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0036] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A charging scheduling method for underwater wireless rechargeable sensor networks based on UWDQN is characterized by: The specific steps are as follows: Step 1: Construct an underwater wireless rechargeable sensor network model including a base station, an underwater mobile charger, and underwater sensor nodes; Step 2: Construct a motion energy consumption model of the underwater mobile charger. The formula of the motion energy consumption model is: ； in, Represents the underwater mobile charger slave node To Node energy consumption, m represents the mass of the underwater mobile charger, g is the acceleration due to gravity, is the density of water, V is the displacement volume of the underwater mobile charger, Representation node and nodes The vertical distance between Indicates the energy consumption of water resistance; Step 3: Construct a node energy consumption model of the underwater sensor node. The formula of the node energy consumption model is: ； in, Representation node The data transmission energy consumption rate, Indicates the energy consumption of receiving 1 bit of data, Indicates that the node The set of other nodes that forward data, express midpoint The amount of data generated, Representation node The data generated per second, Indicates that each bit of data is from the node Energy consumption of transmission to the next hop; Step 4: Construct a charging energy consumption model for the underwater mobile charger; Step 5: Based on the UWDQN method, determine the charging node path of the underwater mobile charger.

2. The charging scheduling method for underwater wireless rechargeable sensor networks based on UWDQN according to claim 1 is characterized in that: The underwater wireless rechargeable sensor network model includes: Establish an underwater three-dimensional coordinate system to determine the location of the base station, the underwater mobile charger, the location of each sensor node, and the routing relationship between each sensor node in the underwater wireless rechargeable sensor network; Determine the energy, charging energy threshold, and movement energy threshold of the underwater mobile charger; Determine the maximum battery capacity, charge request threshold, and death threshold of each sensor node.

3. The charging scheduling method for underwater wireless rechargeable sensor networks based on UWDQN according to claim 1 is characterized in that: The charging energy consumption model includes: Energy consumption during partial charging in mode A, when the node If the remaining power is less than the charge request threshold, , then the amount of charge that the underwater mobile charger provides to the node is: ； in, Representation node The energy consumption rate of data transmission; and They represent the maximum battery capacity and charging request threshold of the node respectively; T Indicates the node charging time; Indicates that the underwater mobile charger is a node The energy consumed for a full or partial charge; Indicates that the underwater mobile charger is a node Energy consumed by partial charging; Representation node The remaining energy; Indicates the energy loss rate of wireless charging; Energy consumption when fully charged in mode B, when and When it is determined that the remaining energy of the underwater mobile charger is sufficient to fully charge, the amount of charge of the node by the underwater mobile charger is: ； in, Indicates that the underwater mobile charger is a node Energy consumed for a full charge; In mode C, when the remaining charging energy of the underwater mobile charger is insufficient, the amount of charging the node by the underwater mobile charger is: ;in, Indicates the remaining charging energy of the underwater mobile charger.

4. The charging scheduling method for underwater wireless rechargeable sensor networks based on UWDQN according to claim 1 is characterized in that: The step 5 comprises: Step 5.1: Determine a charging path representation of the underwater mobile charger based on the underwater wireless rechargeable sensor network model; determine the objectives and constraints of the charging path based on the mobile charger motion energy consumption model and the underwater mobile charger charging energy consumption model; Step 5.2: According to the underwater wireless rechargeable sensor network model, set the state space and action space of the Q network, confirm the iterative update formula of the Q network parameters, set the reward function based on the objectives and constraints, and establish the UWDQN network; Step 5.3: Train the UWDQN network. After the training converges, schedule the underwater mobile charger to perform the charging task according to the optimal action predicted by the UWDQN network.

5. The charging scheduling method for underwater wireless rechargeable sensor networks based on UWDQN according to claim 4 is characterized in that: The charging path is expressed as: ;in, represents a base station, Indicates the last sensor node to be charged; The goal is to maximize the charging efficiency and minimize the death node rate, which can be expressed as: ； ； The constraint is that the charging energy and driving energy consumption are less than the capacity of each battery, which can be expressed as: ； ； in, Indicates the charging efficiency, represents the death node rate; Representation node status, n is the total number of nodes, 1 means Death failure, 0 means Survival; and They represent the charging energy threshold and moving energy threshold of the underwater mobile charger respectively; Indicates the energy consumed by charging. , Indicates driving energy consumption , and Indicates that the underwater mobile charger is a node The energy consumed by charging and driving to the node The energy consumed.

6. The charging scheduling method for underwater wireless rechargeable sensor networks based on UWDQN according to claim 5 is characterized in that: The state space s Including: the location, energy, energy consumption of all nodes, and the location and energy of underwater mobile chargers; the action space Including: the location of all nodes and base stations; The iterative update formula is: ； in, are the parameters of the current neural network, is the learning rate, Represent the loss function as a whole Relative to the parameter The gradient, is the mean square error loss function, Indicates that the current neural network is in state s Next action Estimated value of The target Q value calculated for the target network target network, are the parameters of the target network, Performing actions for immediate rewards The immediate feedback signal obtained from the environment afterwards; is a discount factor used to balance the importance of current rewards and future rewards; As a whole, it represents the target network target network for the next state Maximum Q value estimation; The reward function R The formula is: ； in, and All weights are greater than 0. and Add up to 1.

7. The charging scheduling method for underwater wireless rechargeable sensor networks based on UWDQN according to claim 6 is characterized in that: In the state space and the action space, Underwater mobile charger Energy in tense The formula is: ； in, Indicates that the underwater mobile charger is q Energy in tense, Indicates that the underwater mobile charger has been transferred from the previous node Move to the next node Movement energy consumption; Indicates that the underwater mobile charger is given to the previous node Charging energy consumption; sensor nodes exist Energy in tense The formula is: ； in, Representation node exist q Tense energy, represents the sum of node charging time and travel time, T Charging time for the node, Indicates the energy loss rate of wireless charging, represents the charging efficiency of the underwater mobile charger to charge the node, Representation node The data transmission energy consumption rate, UMC stands for underwater mobile charger.

8. The charging scheduling method for underwater wireless rechargeable sensor networks based on UWDQN according to claim 4 is characterized in that: The step 5.3 specifically includes: Collect underwater mobile charger information and sensor node information, update the underwater wireless rechargeable sensor network model, and determine the initial parameters of the state space and action space of the UWDQN network; pass The strategy selects actions to The probability of randomly selecting a sensor node to be charged and determining the charging amount according to the charging energy consumption model; Intelligent agent The probability of selecting the action with the largest current Q value; the agent obtains the reward and updates the parameters of the Q network, and stores the training sample content of each step of training in the experience replay pool for sampling training; Randomly select a number of training samples from the experience replay pool, perform training and fitting on the UWDQN network, and use the converged UWDQN network to predict the optimal action; The underwater mobile charger is scheduled to charge each sensor node according to the optimal action sequence.

Images