An underwater wireless sensor network coverage optimization method and system

Abstract

This invention discloses a method and system for optimizing coverage in underwater wireless sensor networks, belonging to the field of sensor network optimization technology. The method first establishes a multi-objective optimization model encompassing cooperative coverage, energy consumption, energy balance, and the utilization of water flow dynamics; it constructs the state space, action space, and reward function of a Markov decision process model; and it improves the traditional marine predator algorithm through adaptive parameter adjustment and population diversity stage segmentation to obtain an enhanced marine predator algorithm; it then uses the enhanced marine predator algorithm to optimize the hyperparameters of the near-end strategy optimization algorithm, obtaining the optimal hyperparameter combination; finally, it trains the Markov decision process model using the optimized near-end strategy optimization algorithm to obtain the optimal deployment strategy for underwater wireless sensor network nodes. This method can improve coverage in a three-dimensional underwater dynamic environment, reduce node energy consumption, balance energy distribution, and adapt to complex and dynamic underwater deployment scenarios.

Description

Technical Field

[0001] This application relates to the field of sensor network optimization technology, and in particular to a method and system for optimizing the coverage of underwater wireless sensor networks. Background Technology

[0002] As human activities and scientific research continue to extend into the ocean, global attention and understanding of the ocean are increasing. Underwater wireless sensor networks (UWSNs) are distributed, self-organizing networks composed of numerous sensor nodes deployed underwater, playing a crucial role in marine environmental monitoring, resource exploration, underwater defense, and disaster early warning. Compared to the stable and easily maintained deployment environment on land, underwater sensor nodes are typically deployed in dynamic and complex underwater environments. Therefore, establishing a stable and reliable network under these conditions presents a very significant challenge.

[0003] In real-world scenarios, deterministic deployment of sensor nodes underwater is difficult, so random deployment is typically used. This deployment method leads to numerous coverage blind spots and overlapping areas, necessitating secondary deployment by moving sensor nodes to optimize coverage. However, underwater sensor nodes are usually powered by batteries with limited capacity, and moving them consumes a significant amount of energy, exacerbating the energy constraint problem. Therefore, how to collaboratively plan the movement paths of sensor nodes in complex underwater environments to maximize coverage, minimize total movement energy consumption, and balance energy consumption among sensor nodes has become a highly challenging multi-objective optimization problem.

[0004] Currently, research often employs virtual force algorithms and swarm intelligence algorithms, which can improve coverage to some extent. However, these methods typically do not consider the influence of water flow in the real environment. When faced with dynamic uncertainties such as ocean current disturbances, their adaptability is limited, and they are prone to getting trapped in local optima. Summary of the Invention

[0005] Therefore, it is necessary to provide a method and system for optimizing the coverage of underwater wireless sensor networks to address the aforementioned technical problems.

[0006] The following technical solution is adopted in this specification: This specification provides a method for optimizing coverage of underwater wireless sensor networks, including: A three-dimensional underwater monitoring area with a wireless sensor network deployed is obtained, and a multi-objective optimization model is constructed for the three-dimensional underwater monitoring area with network coverage, total node energy consumption, node energy balance and ocean current utilization rate as optimization objectives. A composite reward function is constructed by using multiple optimization objectives in the multi-objective optimization model as reward and penalty terms, and a Markov decision process model is then constructed. Based on the traditional marine predator algorithm, an enhanced marine predator algorithm is obtained by introducing a cosine adaptive weight factor and a dynamic stage division mechanism based on population diversity. The enhanced marine predator algorithm is used to optimize the hyperparameter combination used to control the near-end strategy optimization algorithm when performing underwater deployment strategy learning. Each hyperparameter combination is treated as a population individual. Based on the population diversity score according to the dynamic stage division mechanism, the enhanced marine predator algorithm is adaptively divided into exploration stage, transition stage and development stage. At each stage, the position of the population individual is iteratively updated using the corresponding position update formula and the cosine adaptive weight factor, and the optimal hyperparameter combination is output. An optimal near-end policy optimization algorithm with optimal hyperparameter combination is used to solve the Markov decision process model with the objective of maximizing the cumulative composite reward, thereby obtaining the optimal deployment strategy for underwater wireless sensor network nodes.

[0007] Furthermore, the construction of the multi-objective optimization model includes: The three-dimensional underwater monitoring area is uniformly divided into grids. The probability of each sensor node sensing a grid point is calculated using a probabilistic sensing model. The network coverage rate is obtained by determining whether the grid point is covered based on the joint sensing probability of all nodes sensing the same grid point and a preset detection threshold. Set the energy consumption per unit distance traveled. Calculate the energy consumption of each sensor node based on the cumulative distance traveled from its initial position to its current position. Sum the energy consumption of all sensor nodes as the total energy consumption of the nodes. Calculate the average energy consumption of all sensor nodes, and obtain the standard deviation of energy consumption by taking the square root of the sum of the squares of the differences between the energy consumption of each sensor node and the average value, which serves as the node energy balance. Obtain the three-dimensional velocity components of the ocean current at the location of the sensor node and the unit vector of the sensor node's navigation direction. Calculate the ratio of the projection value of the ocean current velocity in the navigation direction to the amplitude of the ocean current velocity, which is used as the ocean current utilization rate of the sensor node. The average value of the ocean current utilization rates of all sensor nodes is used as the overall ocean current utilization rate. The network coverage, total node energy consumption, node energy balance, and ocean current utilization rate are used as optimization objectives to construct a multi-objective optimization model for the three-dimensional underwater monitoring area.

[0008] Furthermore, a composite reward function is constructed by using multiple optimization objectives in the multi-objective optimization model as reward and penalty terms, including: Obtain the network coverage value at the current moment and use the value as a coverage bonus. Get the network coverage at the current time and the previous time, and use the difference in network coverage between the two times as the coverage change reward; The ratio of the total energy consumption of the sensor nodes to the sum of the initial energy of all nodes is used as the energy consumption penalty. Calculate the average energy consumption of all sensor nodes, and use the ratio of the node energy balance to the average energy consumption as the energy balance penalty; The average ocean current utilization rate of all nodes is used as the ocean current utilization reward; A composite reward function is obtained by weighted summing of the coverage reward, coverage change reward, sensor energy consumption penalty, energy balance penalty, and ocean current utilization rate reward; wherein the sum of the weight coefficients corresponding to the rewards and penalties in the composite reward function is 1.

[0009] Furthermore, the construction process of the Markov decision process model includes: The state space of the Markov decision process model is composed of the three-dimensional spatial coordinates of all sensor nodes, the cumulative mobile energy consumption of each node, the real-time ocean current velocity at the location of each node, and the current overall network coverage. The continuous action space of the Markov decision process model is composed of the displacement components of all sensor nodes on the three-dimensional coordinate axes. The composite reward function of the Markov decision process model is obtained by weighted fusion of multiple optimization objectives in the multi-objective optimization model.

[0010] Furthermore, by introducing a cosine adaptive weighting factor and a dynamic stage division mechanism based on population diversity, an enhanced marine predator algorithm is obtained, including: Based on the traditional marine predation algorithm, a cosine adaptive weighting factor ω is introduced; Based on the dynamic stage division mechanism of population diversity, the center value of each dimension in each population is calculated, and the Euclidean distance between each individual and the population center is obtained as the diversity value of each individual. The diversity values ​​of all individuals are normalized to obtain the diversity score. Based on the diversity score, the ocean predator algorithm is adaptively divided into exploration stage, transition stage and development stage to obtain the enhanced ocean predator algorithm. The weighting factor ω is dynamically adjusted according to the algorithm stage of the enhanced marine predator algorithm. The formula for calculating the weighting factor is expressed as follows: ω = 0.5 × (1 + cos(π × t / T)); Where t is the current iteration number and T is the maximum iteration number.

[0011] Furthermore, the optimization process for the optimal hyperparameter combination includes: The population size, maximum number of iterations, short training steps, and number of evaluation rounds of the enhanced marine predator algorithm are initialized, and each individual in the population is used as a set of hyperparameters for a proximal policy optimization algorithm; the hyperparameters include learning rate, discount factor, generalized dominance estimation parameter, pruning range, and entropy coefficient. For each set of hyperparameter combinations, the proximal policy optimization agent is used to complete a short-term training of a preset number of steps in a three-dimensional underwater environment through an initialization process. After training, the average reward is obtained through multiple rounds of evaluation, and the average reward is used as the fitness value of the individual in the population. Calculate the population diversity score and adaptively divide the stages of the enhanced marine predator algorithm according to the diversity score: when the diversity score is less than 1 / 3, execute the exploration stage; when the diversity score is between 1 / 3 and 2 / 3, execute the transition stage; and when the diversity score is greater than 2 / 3, execute the development stage. Based on the disturbance mechanism of the fish gathering device, the population is disturbed in an environment with a set probability to maintain population diversity. The elite predators are iteratively updated based on their fitness values, and the hyperparameter combination with the optimal fitness is output, thus completing the optimization of key hyperparameters in the near-end strategy optimization algorithm.

[0012] Furthermore, with the objective of maximizing the cumulative compound reward, the Markov decision process model is solved, including: Initialize the policy network parameters θ and value function parameters φ of the near-end policy optimization algorithm, and clear the empirical trajectory buffer; In each training round, the agent selects an action based on the current state, observes the next state and reward after executing the action, and stores the transition sample (s(t), a(t), R(t), s(t+1)) in the buffer. The generalized advantage estimation is used to calculate the advantage function. The difference between the old and new policies is constrained by pruning the surrogate objective function. The policy network is updated in K epochs. Minimize the temporal difference error to update the value function, repeat the training until reward convergence is achieved, and obtain the optimal solution of the Markov decision process model, which is the optimal deployment strategy for underwater wireless sensor network nodes.

[0013] This specification provides a multi-target coverage optimization system for underwater wireless sensor networks, including: Multi-objective model construction module: used to acquire a three-dimensional underwater monitoring area with a wireless sensor network deployed, and to construct a multi-objective optimization model for the three-dimensional underwater monitoring area with network coverage, total node energy consumption, node energy balance and ocean current utilization rate as optimization objectives; Markov Decision Process Construction Module: Used to construct a composite reward function by taking multiple optimization objectives in the multi-objective optimization model as reward and penalty terms, so as to construct the Markov decision process model; Algorithm Improvement Module: This module is used to improve the traditional marine predator algorithm by introducing a cosine adaptive weighting factor and a dynamic stage division mechanism based on population diversity, resulting in an enhanced marine predator algorithm. Hyperparameter optimization module: This module is used to optimize the hyperparameter combination used to control the near-end strategy optimization algorithm when performing underwater deployment strategy learning using the enhanced marine predator algorithm. Each hyperparameter combination is treated as a population individual. Based on the population diversity score and a dynamic stage division mechanism, the enhanced marine predator algorithm is adaptively divided into exploration, transition, and development stages. The position of the population individual is iteratively updated using the corresponding position update formula and the cosine adaptive weight factor at each stage, and the optimal hyperparameter combination is output. The strategy training module is used to solve the Markov decision process model by employing a near-end policy optimization algorithm with the optimal hyperparameter combination, aiming to maximize the cumulative composite reward, and obtain the optimal deployment strategy for underwater wireless sensor network nodes.

[0014] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described underwater wireless sensor network coverage optimization method.

[0015] This specification provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described underwater wireless sensor network coverage optimization method.

[0016] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects: In the underwater wireless sensor network coverage optimization method provided in this specification, firstly, a multi-objective optimization model is established, and a Markov decision process model is constructed. Then, the hyperparameters of the near-end strategy optimization algorithm are optimized using an enhanced marine predator algorithm to obtain the optimal hyperparameter combination. Finally, the optimized near-end strategy optimization algorithm is used to solve the Markov decision process, yielding the optimal deployment strategy for underwater wireless sensor network nodes. Specifically, by introducing a cosine adaptive weighting factor and a dynamic stage partitioning mechanism based on population diversity, the enhanced marine predator algorithm optimizes the hyperparameters of the near-end strategy optimization algorithm. This ensures that the final deployment strategy effectively avoids sensor node clustering and coverage blind spots, effectively improves the coverage of the three-dimensional underwater dynamic environment, reduces sensor node energy consumption and balances energy distribution, and adapts to complex and dynamic underwater deployment scenarios. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 A schematic flowchart of an underwater wireless sensor network coverage optimization method in three-dimensional dynamic space provided by the present invention; Figure 2 This is a schematic diagram illustrating the planned navigation direction of a sensor node and the ocean current speed, provided by the present invention. Figure 3 A schematic diagram of the sensing range of a sensor node provided by the present invention; Figure 4 A flowchart illustrating an enhanced marine predator algorithm provided by this invention; Figure 5 This is a schematic diagram of the displacement of a sensor node in the monitoring area. Figure 6 This is a schematic diagram illustrating the coverage effect of a sensor node over a monitored area. Figure 7 This is a schematic diagram showing the change in coverage of the method of this invention and the conventional method as a function of iteration; Figure 8 A schematic diagram of an underwater wireless sensor network coverage optimization system in a three-dimensional dynamic space provided by the present invention; Figure 9 This invention provides a schematic diagram of a computer device for implementing a method for optimizing underwater wireless sensor network coverage in a three-dimensional dynamic space. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.

[0020] Currently, most coverage optimization research focuses on two-dimensional regions. Coverage strategies for two-dimensional environments cannot be directly applied to dynamic and complex three-dimensional underwater environments. Furthermore, coverage optimization problems for three-dimensional environments are mostly conducted in ideal three-dimensional space, without considering underwater dynamic factors such as ocean current disturbances. This results in significant shortcomings when sensing complex underwater environments and fails to meet the monitoring needs of dynamic three-dimensional environments.

[0021] The prior art mentions a 3D sensor network deployment strategy based on an improved swarm intelligence algorithm. Its basic principle is to enhance the algorithm's global search capability and population diversity by improving the algorithm's search mechanism or mutation operation, thereby avoiding the algorithm from getting trapped in local optima and improving the coverage optimization effect in the 3D environment.

[0022] The existing 3D node deployment strategies mentioned above mostly employ a single heuristic algorithm or a traditional deep reinforcement learning algorithm. However, these methods have significant drawbacks when dealing with dynamic 3D underwater scenarios: heuristic algorithms, based on simplified environmental assumptions, are poorly adaptable to dynamic uncertainties such as ocean current disturbances; traditional deep reinforcement learning algorithms are extremely sensitive to hyperparameter configuration, and unreasonable parameter settings can easily lead to training instability or even convergence difficulties, and they lack the design of state, action, and reward functions specifically for dynamic underwater environments, resulting in poor strategy adaptability. In summary, existing optimization strategies suffer from low network coverage, high energy consumption, uneven energy distribution, and insufficient algorithm adaptability, and their core algorithm's optimization capabilities and adaptability to dynamic environments need to be improved.

[0023] Based on this, the present invention provides a method and system for optimizing the coverage of underwater wireless sensor networks in a three-dimensional dynamic space, which achieves higher network coverage with lower energy consumption and better energy balance, thereby optimizing network coverage and network lifetime.

[0024] The technical solutions provided by the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0025] Figure 1 This is a schematic diagram of a method for optimizing underwater wireless sensor network coverage in a three-dimensional dynamic space according to the present invention, which specifically includes the following steps: S101, Construct a multi-objective optimization model that integrates ocean current sensing, wherein the model objectives include coverage, total node energy consumption, node energy balance, and ocean current utilization.

[0026] First, the three-dimensional underwater monitoring area is uniformly divided into grids, with each grid vertex being a grid point. The sensor nodes adopt a probabilistic sensing model, such as... Figure 2 As shown. Assuming the three-dimensional underwater monitoring area is discretized into M grid points, the set of grid points is represented as P = {P1, P2, ..., P...}. M The set of sensor nodes deployed in the monitoring area is represented as S = {S1, S2, ..., S}. N For any point P within the monitoring area... j ∈P, sensor node S i The probability of perceiving a point ∈S is represented by P(S). i P j The formula is as follows:

[0027] in, , R s This represents the sensing radius of the sensor node. R e Indicates an uncertain sensing range. λ It is a proportionality coefficient. λ = R e −R s +d ( S i ，P j ), α = R s +R e −d ( S i ，P j ), β It is the decay index.

[0028] Based on the distance of each sensor node to the grid point, the overall perception probability of each grid point is obtained: in, N This refers to the number of nodes. Set the detection threshold. C th ,if C(P j )≥C th If so, it is considered that the sensor network has effectively covered the area. P j .

[0029] And determine the coverage rate based on the overall perception probability of each grid point: Where M represents the number of grid points in the monitoring area.

[0030] The energy consumption of sensor nodes is mainly caused by their movement, significantly exceeding the energy consumption during sensing and communication processes. Therefore, the energy consumption model primarily focuses on the energy consumption during sensor node movement. Let the energy consumption per unit distance movement of the sensor node be e0, then the energy consumption of sensor node S... i Mobile energy consumption is expressed as follows:

[0031] Among them, E ini d represents the initial energy of the sensor node. i This represents the distance traveled during deployment by the sensor node itself.

[0032] The energy balance among sensor nodes is represented by the standard deviation of energy consumption, as shown below: Among them, E avg This represents the average energy consumption of all nodes.

[0033] Ocean currents are a key factor influencing the energy consumption of sensor nodes. If a sensor node moves in the same direction as the ocean current, the current will help it reach its destination with lower energy consumption. Therefore, an ocean current utilization model was established to maximize the contribution of the ocean current to the sensor node's mobility by aligning the sensor node's motion direction with the ocean current direction as much as possible. The ocean current utilization rate is defined as the ratio of the projection amplitude of the ocean current velocity and the sensor node's own velocity to the amplitude of the ocean current velocity, and its calculation formula is as follows:

[0034] Among them, v c，|| This represents the projection of the ocean current velocity vector onto the unit vector of the planned course, while ||v c || indicates the speed of the ocean current.

[0035] like Figure 3 As shown, θ represents the self-velocity v of the node. self i and ocean current speed v c The angle between them. When θ < 90° ◦ At that time, the node moves downstream, and the ocean current utilization rate is greater than 0.

[0036] S102, Construct a Markov decision process model.

[0037] To achieve optimal deployment of underwater sensor nodes, they are modeled as Markov decision processes, and a quintuple (S, A, R, P, γ) is defined, where S is the state space, A is the action space, R is the reward function, P is the state transition probability, and γ is the discount factor.

[0038] Define a state space, which consists of the three-dimensional spatial coordinates of all sensor nodes, the cumulative mobile energy consumption of each sensor node, the real-time ocean current velocity at the location of each sensor node, and the current overall network coverage. The formula is: in,( x i 、yi 、z i This indicates the location and energy consumption of the sensor node. E i 、V c i Let i represent the ocean current velocity at the location of sensor node i, and Cov represent the coverage at time step t.

[0039] The motion space adopts a continuous motion space, which consists of the displacement components of all sensor nodes on the three-dimensional coordinate axes, and the formula is: The reward function is a composite reward function, designed based on the aforementioned multi-objective optimization model. It consists of a weighted sum of coverage reward, coverage change reward, energy consumption penalty, energy balance penalty, and ocean current utilization reward. The sum of the weight coefficients corresponding to each reward and penalty is 1. The formula is as follows: in, This represents the weighting coefficients, which sum to 1.

[0040] Optional components of the composite reward function include: The coverage bonus is calculated based on the network coverage value at the current moment, using the following formula: The coverage change reward is calculated as the difference between the network coverage at the current time and the previous time, using the following formula: To discourage reckless movement and encourage energy conservation, energy consumption is penalized. The energy penalty is the ratio of the sum of the cumulative energy consumption of all sensor nodes to the sum of the initial energy of all sensor nodes, as shown in the formula:

[0041] Energy balance penalties prevent deployment strategies where some sensor nodes move excessively while others remain unchanged, thus extending network lifetime. The energy balance penalty is the ratio of the standard deviation of energy consumption to the average energy consumption of sensor nodes.

[0042] The ocean current utilization incentive aims to reduce energy consumption by encouraging movement directions that utilize ocean current assistance. The ocean current utilization incentive is calculated as the average of the ocean current utilization rates across all sensor nodes.

[0043] S103, an enhanced marine predator algorithm is obtained by improving the original marine predator algorithm, including: To address the issues of low search efficiency caused by the original Marine Predator algorithm's tendency to get trapped in local optima and its inability to adapt to the diverse deployment scenarios of underwater wireless sensor networks when searching for hyperparameters for near-end policy optimization, an improvement is made to the original Marine Predator algorithm. A cosine adaptive weight factor ω is introduced to balance the exploration of unknown parameter ranges and the development of known ranges during the hyperparameter search process for near-end policy optimization. This weight factor is dynamically adjusted with the iteration process, and its calculation formula is as follows:

[0044] Where t is the current iteration number and T is the maximum iteration number.

[0045] A dynamic stage partitioning mechanism based on population diversity is constructed, where the hyperparameter combination of each proximal strategy optimization algorithm is encoded as a search individual in the marine predator algorithm, and the dimension of the marine predator algorithm is the number of hyperparameters. The population centroid value of all individuals in each dimension is calculated:

[0046] Where xi,j represents the j-th dimension of the i-th individual, and P is the population size.

[0047] The Euclidean distance between an individual and the population center is used as the diversity value, and the formula is: Where d represents dimension.

[0048] After obtaining the diversity values ​​for all individuals, the extreme values ​​and the population diversity value are determined. The population diversity value is represented by the average population diversity, as follows: Normalization yields a diversity score, which reflects the degree of difference in node deployment strategies trained by the proximal strategy optimization algorithm under different hyperparameter combinations. The diversity score is then adaptively used to divide the search, transition, and development phases to optimize the ocean predator algorithm's search strategy in the hyperparameter space.

[0049] Where ϵ is the smallest positive number. p i Within the range [0, 1]. When p i When p < 1 / 3, the algorithm enters the development phase, performing a fine-grained search in the local hyperparameter region; when p < 1 / 3, the algorithm continues its development. i When p < 2 / 3, the algorithm enters the transition phase; when p i When the result is greater than 2 / 3, the algorithm enters the exploration phase, which involves a large-scale search in the hyperparameter space to avoid missing the optimal combination of hyperparameters for deploying underwater sensor nodes.

[0050] S104 utilizes the enhanced marine predator algorithm to optimize key hyperparameters of the near-end strategy optimization algorithm, including: Because the hyperparameters of near-end strategy optimization algorithms are highly sensitive to environmental parameters such as ocean currents and sensor node density in underwater sensor network coverage optimization, an enhanced marine predator algorithm is introduced to optimize the hyperparameters of the near-end strategy optimization algorithm. For example... Figure 4 As shown, the population size, maximum number of iterations, short training steps, and number of evaluation rounds of the enhanced marine predator algorithm are initialized. Each individual in the population is assigned a set of key hyperparameters of the proximal policy optimization algorithm. The hyperparameters include learning rate, discount factor, generalized dominance estimation parameters, pruning range, and entropy coefficient. The search range of the hyperparameters is set according to the characteristics of the underwater wireless sensor network.

[0051] For each hyperparameter combination, the corresponding proximal policy optimization agent is initialized, enabling it to complete a short training period of a set number of steps in a three-dimensional underwater environment. The agent selects actions based on the state of the Markov decision process, receives a reward after executing the action, and updates the policy based on the reward. The reward is a weighted value of coverage, sensor node energy consumption, sensor node energy consumption balance, and ocean current utilization. After training, the average reward is calculated through multiple rounds of evaluation, and this average reward is used as the fitness value of the corresponding individual in the population. Calculate the population diversity score, adaptively divide the algorithm into stages based on the diversity score, and execute the corresponding position update formula to update the hyperparameters of the near-end policy optimization algorithm. When the diversity score is less than one-third, the search range is expanded using an exploration position update formula, which is:

[0052] Here, the symbol ⊗ represents term-by-term multiplication, and R B Let R represent a random vector that follows Brownian motion, where P is a constant coefficient and R is a random vector that is uniformly distributed in the range [0, 1].

[0053] When the diversity score is between one-third and two-thirds, a transitional phase position update formula is used to balance exploration and development: The population is divided into two parts: the first part consists of prey that explores using a levy strategy, while the second part consists of predators that utilize Brownian motion for hunting. L It is a Levy-compliant random vector, and CF is an adaptive control parameter, defined as follows: When the diversity score is greater than two-thirds, a local fine-grained search is performed using the development location update formula: Based on the perturbation mechanism of the fish gathering device, environmental disturbances are performed on the population with a set probability, and corresponding location update formulas are executed to maintain population diversity: Where r is a random number and r∈[0,1], U is a random binary vector, and Prey r1 (t) and Prey r2 X(t) are two distinct individuals randomly selected in the t-th iteration. max and X min It is a vector composed of the upper and lower bounds of all dimensions, and FADs are the perturbation probability thresholds.

[0054] The elite predators are updated based on their fitness values. The above population position update, fitness evaluation, and elite update process is iteratively executed until the maximum number of iterations is reached. The hyperparameter combination with the best fitness is then output, thus obtaining the optimal hyperparameter combination suitable for underwater wireless sensor networks.

[0055] S105, Based on the optimized hyperparameter training of the proximal policy optimization algorithm, solve the Markov decision process to maximize the cumulative reward and obtain the optimal deployment strategy for the sensor nodes, including: Initialize the policy network parameters θ and value function parameters φ of the near-end policy optimization algorithm, and clear the empirical trajectory buffer; In each training round, the agent selects an action based on the current state, observes the next state and reward after executing the action, and stores the transition sample (s(t), a(t), R(t), s(t+1)) in the buffer. The generalized advantage estimation is used to calculate the advantage function. The difference between the old and new policies is constrained by pruning the surrogate objective function. The policy network is updated in K epochs. Minimize the temporal difference error to update the value function, and repeat the training until the set number of rounds or reward convergence is reached.

[0056] This invention employs an optimization strategy combining an enhanced marine predator algorithm and a proximal policy optimization algorithm, redesigning the state space, action space, and reward function of the Markov decision process. Secondly, it improves the traditional marine predator algorithm through adaptive parameter adjustment and population diversity stage partitioning, resulting in an enhanced marine predator algorithm. This enhanced marine predator algorithm is then used to optimize the hyperparameters of the proximal policy optimization algorithm, yielding the optimal hyperparameter combination. Finally, the enhanced marine predator algorithm is used again to optimize the hyperparameters of the proximal policy optimization algorithm, obtaining the optimal hyperparameter combination. The optimized proximal policy optimization algorithm is then used to solve the Markov decision process, resulting in the optimal deployment strategy for underwater wireless sensor network nodes. This invention effectively improves the coverage of three-dimensional underwater dynamic environments, reduces sensor node energy consumption, balances energy distribution, and adapts to complex and dynamic underwater deployment scenarios.

[0057] To further illustrate the above issues, a simulation is performed on the underwater wireless sensor network coverage problem using the optimization strategy based on the Enhanced Marine Predators Proximal Policy Optimization (EMPPO) algorithm employed in this invention. Consider 50 sensors deployed in an area of ​​100m × 100m × 100m, with a sensor sensing radius R... S =30m.

[0058] When the monitored area is 100m×100m×100m and there are 50 sensor nodes, the displacement of the sensor nodes is as follows: Figure 5 As shown, the black solid circle represents the initial position of the sensor node, the red solid circle represents the final position of the sensor node, and the connecting line represents the linear displacement between the two. It can be observed that after EMPPO optimization, the sensor nodes maintain a uniform distribution throughout the monitoring area, without obvious clustering, demonstrating strong coverage optimization capabilities. Furthermore, the movement distance is moderate, without excessively long or short movements, indicating that the algorithm effectively controls the movement amplitude during deployment. The coverage effect is as follows: Figure 6 As shown. From Figure 6 The distribution of sensor nodes shows that after executing EMPPO, the sensor node distribution is more uniform and the coverage effect is better.

[0059] To compare the convergence performance of the algorithms, the coverage iterations of EMPPO were compared with those of the Multi-objective Spider Wasp Optimizer (MOSWO), the 3-D virtual force algorithm (3DVFA), and the Proximal Policy Optimization (PPO) algorithm. Figure 7 As shown, the horizontal axis represents the number of iterations, and the vertical axis represents the coverage rate. Figure 7 As can be seen, EMPPO's convergence speed is faster and its coverage is significantly improved compared to the previous version.

[0060] Table 1 compares the total energy consumption, maximum sensor node energy consumption, minimum sensor node energy consumption, and energy consumption standard deviation of the four algorithms. EMPPO not only achieved the highest coverage but also had the lowest total energy consumption, approximately 7989.90 joules. 3DVFA's total energy consumption was 14869.46 joules, significantly exceeding EMPPO. PPO fell between EMPPO and 3DVFA, with a total energy consumption of 11830.15 joules. This confirms that EMPA can identify the optimal hyperparameters for PPO, making EMPPO significantly superior to the original PPO in terms of energy efficiency. Furthermore, its low energy consumption is attributed to its efficient sensor node movement path planning. During deployment, ocean currents were treated as an available driving force rather than an obstacle to overcome, thus avoiding unnecessary energy consumption. In terms of energy consumption standard deviation, EMPPO achieved 76.2, significantly lower than MOSWO (590.31) and 3DVFA (162.09). This indicates that, compared to traditional metaheuristic algorithms, the EMPPO algorithm distributes mobile energy more evenly among sensor nodes and exhibits better energy balance. Furthermore, the energy consumption standard deviation of EMPPO is lower than that of PPO (108.19), suggesting that EMPA-based hyperparameter optimization effectively improves the stability of PPO.

[0061] Table 1. Comparison of energy consumption and standard deviation of energy consumption for four different algorithms. When applying the underwater wireless sensor network coverage optimization method in three-dimensional dynamic space provided by this invention, it is not necessary to consider... Figure 1 The steps shown are executed in sequence. The specific execution order of each step can be determined as needed, and this invention does not impose any restrictions on it.

[0062] The above describes a method for optimizing underwater wireless sensor network coverage in a three-dimensional dynamic space, provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding system for optimizing underwater wireless sensor network coverage in a three-dimensional dynamic space, such as... Figure 8 As shown.

[0063] Figure 8 A schematic diagram of an underwater wireless sensor network coverage optimization system in a three-dimensional dynamic space provided by the present invention is shown. The system includes: The multi-objective model construction module constructs a multi-objective optimization model that integrates ocean current sensing. The model objectives include coverage, total energy consumption of sensor nodes, energy balance of sensor nodes, and ocean current utilization rate.

[0064] The Markov Decision Process (MDP) construction module builds a MDP model and defines the state space, action space, and composite reward function to suit underwater deployment scenarios.

[0065] The hyperparameter optimization module improves the traditional marine predator algorithm by introducing a cosine adaptive weighting factor and a dynamic stage division mechanism based on population diversity to obtain an enhanced marine predator algorithm. The enhanced marine predator algorithm is used to optimize the key hyperparameters of the near-end strategy optimization algorithm. The hyperparameters include the learning rate α, discount factor γ, generalized dominance estimation parameter λ, pruning range ε, and entropy coefficient β.

[0066] The strategy training module trains a near-end strategy optimization algorithm based on optimized hyperparameters. This algorithm learns the optimal deployment strategy by interacting with the underwater environment until the set number of iterations is reached or the convergence condition is met.

[0067] Optionally, the system may also include a data processing module and a control and management module.

[0068] The data processing module is used to receive and import three-dimensional terrain data, sensor node parameters, and grid point distribution information of the monitoring area, and divide the terrain data into grid areas of a specific size to provide a basis for subsequent coverage calculation.

[0069] The control and management module is used to manage the number of iterations and termination conditions of the optimization algorithm, ensuring that the algorithm runs within the predetermined number of iterations and recording the optimal solution for each iteration.

[0070] Specific limitations regarding the underwater wireless sensor network coverage optimization device in three-dimensional dynamic space can be found in the limitations of the underwater wireless sensor network coverage optimization method in three-dimensional dynamic space described above, and will not be repeated here. Each module in the aforementioned underwater wireless sensor network coverage optimization device in three-dimensional dynamic space can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0071] The present invention also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 A method for optimizing underwater wireless sensor network coverage in a three-dimensional dynamic space is provided.

[0072] The present invention also provides Figure 9 The schematic diagram of the computer device shown illustrates that, at the hardware level, the device includes a processor, internal bus, network interface, memory, and non-volatile storage, and may also include other hardware required for various operations. The processor reads the corresponding computer program from the non-volatile storage into memory and then executes it to achieve the above-mentioned functions. Figure 1 A method for optimizing underwater wireless sensor network coverage in a three-dimensional dynamic space is provided.

[0073] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0074] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. An underwater wireless sensor network coverage optimization method, characterized in that, include: A three-dimensional underwater monitoring area with a wireless sensor network deployed is obtained, and a multi-objective optimization model is constructed for the three-dimensional underwater monitoring area with network coverage, total node energy consumption, node energy balance and ocean current utilization rate as optimization objectives. A composite reward function is constructed by using multiple optimization objectives in the multi-objective optimization model as reward and penalty terms, and a Markov decision process model is then constructed. Based on the traditional marine predator algorithm, an enhanced marine predator algorithm is obtained by introducing a cosine adaptive weight factor and a dynamic stage division mechanism based on population diversity. The enhanced marine predator algorithm is used to optimize the hyperparameter combination used to control the near-end strategy optimization algorithm when performing underwater deployment strategy learning. Each hyperparameter combination is treated as a population individual. Based on the population diversity score according to the dynamic stage division mechanism, the enhanced marine predator algorithm is adaptively divided into exploration stage, transition stage and development stage. At each stage, the position of the population individual is iteratively updated using the corresponding position update formula and the cosine adaptive weight factor, and the optimal hyperparameter combination is output. An optimal near-end policy optimization algorithm with optimal hyperparameter combination is used to solve the Markov decision process model with the objective of maximizing the cumulative composite reward, thereby obtaining the optimal deployment strategy for underwater wireless sensor network nodes.

2. The underwater wireless sensor network coverage optimization method of claim 1, wherein, The construction of the multi-objective optimization model includes: The three-dimensional underwater monitoring area is uniformly divided into grids. The probability of each sensor node sensing a grid point is calculated using a probabilistic sensing model. The network coverage rate is obtained by determining whether the grid point is covered based on the joint sensing probability of all nodes sensing the same grid point and a preset detection threshold. Set the energy consumption per unit distance traveled. Calculate the energy consumption of each sensor node based on the cumulative distance traveled from its initial position to its current position. Sum the energy consumption of all sensor nodes as the total energy consumption of the nodes. Calculate the average energy consumption of all sensor nodes, and obtain the standard deviation of energy consumption by taking the square root of the sum of the squares of the differences between the energy consumption of each sensor node and the average value, which serves as the node energy balance. Obtain the three-dimensional velocity components of the ocean current at the location of the sensor node and the unit vector of the sensor node's navigation direction. Calculate the ratio of the projection value of the ocean current velocity in the navigation direction to the amplitude of the ocean current velocity, which is used as the ocean current utilization rate of the sensor node. The average value of the ocean current utilization rates of all sensor nodes is used as the overall ocean current utilization rate. The network coverage, total node energy consumption, node energy balance, and ocean current utilization rate are used as optimization objectives to construct a multi-objective optimization model for the three-dimensional underwater monitoring area.

3. The method of claim 1, wherein, A composite reward function is constructed by using multiple optimization objectives in the multi-objective optimization model as reward and penalty terms, including: Obtain the network coverage value at the current moment and use the value as a coverage bonus. Get the network coverage at the current time and the previous time, and use the difference in network coverage between the two times as the coverage change reward; The ratio of the total energy consumption of the sensor nodes to the sum of the initial energy of all nodes is used as the energy consumption penalty. Calculate the average energy consumption of all sensor nodes, and use the ratio of the node energy balance to the average energy consumption as the energy balance penalty; The average ocean current utilization rate of all nodes is used as the ocean current utilization reward; A composite reward function is obtained by weighted summing of the coverage reward, coverage change reward, sensor energy consumption penalty, energy balance penalty, and ocean current utilization rate reward; wherein the sum of the weight coefficients corresponding to the rewards and penalties in the composite reward function is 1.

4. The underwater wireless sensor network coverage optimization method as described in claim 3, characterized in that, The construction process of the Markov decision process model includes: The state space of the Markov decision process model is composed of the three-dimensional spatial coordinates of all sensor nodes, the cumulative mobile energy consumption of each node, the real-time ocean current velocity at the location of each node, and the current overall network coverage. The continuous action space of the Markov decision process model is composed of the displacement components of all sensor nodes on the three-dimensional coordinate axes. The composite reward function of the Markov decision process model is obtained by weighted fusion of multiple optimization objectives in the multi-objective optimization model.

5. The underwater wireless sensor network coverage optimization method as described in claim 1, characterized in that, The enhanced marine predator algorithm, obtained by introducing a cosine adaptive weighting factor and a dynamic stage division mechanism based on population diversity, includes: Based on the traditional marine predation algorithm, a cosine adaptive weighting factor ω is introduced; Based on the dynamic stage division mechanism of population diversity, the center value of each dimension in each population is calculated, and the Euclidean distance between each individual and the population center is obtained as the diversity value of each individual. The diversity values ​​of all individuals are normalized to obtain the diversity score. Based on the diversity score, the ocean predator algorithm is adaptively divided into exploration stage, transition stage and development stage to obtain the enhanced ocean predator algorithm. The weighting factor ω is dynamically adjusted according to the algorithm stage of the enhanced marine predator algorithm. The formula for calculating the weighting factor is expressed as follows: ω = 0.5 × (1 + cos(π × t / T)); Where t is the current iteration number and T is the maximum iteration number.

6. The underwater wireless sensor network coverage optimization method as described in claim 1, characterized in that, The optimization process for the optimal hyperparameter combination includes: The population size, maximum number of iterations, short training steps, and number of evaluation rounds of the enhanced marine predator algorithm are initialized, and each individual in the population is used as a set of hyperparameters for a proximal policy optimization algorithm; the hyperparameters include learning rate, discount factor, generalized dominance estimation parameter, pruning range, and entropy coefficient. For each set of hyperparameter combinations, the proximal policy optimization agent is used to complete a short-term training of a preset number of steps in a three-dimensional underwater environment through an initialization process. After training, the average reward is obtained through multiple rounds of evaluation, and the average reward is used as the fitness value of the individual in the population. Calculate the population diversity score and adaptively divide the stages of the enhanced marine predator algorithm according to the diversity score: when the diversity score is less than 1 / 3, execute the exploration stage; when the diversity score is between 1 / 3 and 2 / 3, execute the transition stage; and when the diversity score is greater than 2 / 3, execute the development stage. Based on the disturbance mechanism of the fish gathering device, the population is disturbed in an environment with a set probability to maintain population diversity. The elite predators are iteratively updated based on their fitness values, and the hyperparameter combination with the optimal fitness is output, thus completing the optimization of key hyperparameters in the near-end strategy optimization algorithm.

7. The underwater wireless sensor network coverage optimization method as described in claim 1, characterized in that, Solving the Markov decision process model with the objective of maximizing the cumulative compound reward includes: Initialize the policy network parameters θ and value function parameters φ of the near-end policy optimization algorithm, and clear the empirical trajectory buffer; In each training round, the agent selects an action based on the current state, observes the next state and reward after executing the action, and stores the transition sample (s(t), a(t), R(t), s(t+1)) in the buffer. The generalized advantage estimation is used to calculate the advantage function. The difference between the old and new policies is constrained by pruning the surrogate objective function. The policy network is updated in K epochs. Minimize the temporal difference error to update the value function, repeat the training until reward convergence is achieved, and obtain the optimal solution of the Markov decision process model, which is the optimal deployment strategy for underwater wireless sensor network nodes.

8. A multi-target coverage optimization system for underwater wireless sensor networks, characterized in that, include: Multi-objective model construction module: used to acquire a three-dimensional underwater monitoring area with a wireless sensor network deployed, and to construct a multi-objective optimization model for the three-dimensional underwater monitoring area with network coverage, total node energy consumption, node energy balance and ocean current utilization rate as optimization objectives; Markov Decision Process Construction Module: Used to construct a composite reward function by taking multiple optimization objectives in the multi-objective optimization model as reward and penalty terms, so as to construct the Markov decision process model; Algorithm Improvement Module: This module is used to improve the traditional marine predator algorithm by introducing a cosine adaptive weighting factor and a dynamic stage division mechanism based on population diversity, resulting in an enhanced marine predator algorithm. Hyperparameter optimization module: This module is used to optimize the hyperparameter combination used to control the near-end strategy optimization algorithm when performing underwater deployment strategy learning using the enhanced marine predator algorithm. Each hyperparameter combination is treated as a population individual. Based on the population diversity score and a dynamic stage division mechanism, the enhanced marine predator algorithm is adaptively divided into exploration, transition, and development stages. The position of the population individual is iteratively updated using the corresponding position update formula and the cosine adaptive weight factor at each stage, and the optimal hyperparameter combination is output. The strategy training module is used to solve the Markov decision process model by employing a near-end policy optimization algorithm with the optimal hyperparameter combination, aiming to maximize the cumulative composite reward, and obtain the optimal deployment strategy for underwater wireless sensor network nodes.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 7.

10. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any one of claims 1 to 7.

Images