Wireless sensor network clustering routing method and system based on hybrid intelligent optimization

Abstract

The invention relates to a wireless sensor network clustering routing method and system based on hybrid intelligent optimization. The method comprises the following steps: constructing a clustering routing model; on the basis of a clustering routing model, considering the unbalance of node energy consumption, and constructing a network energy consumption model; constructing a joint optimization problem of cluster head selection and a routing path by taking maximization of network comprehensive efficiency as a target based on a network energy consumption model; converting a joint optimization problem into a Markov decision process problem according to the dynamic variability of the network state; a hybrid intelligent optimization framework fusing meta-heuristic and deep reinforcement learning is adopted to solve a Markov decision process problem, and an optimal solution of the problem is obtained, namely, an optimal clustering scheme and a data transmission path are obtained. According to the method, a hybrid algorithm framework fusing grey wolf optimization and the deep Q network is provided, after the joint optimization problem is solved, the life cycle of the wireless sensor network can be remarkably prolonged, the convergence speed is high, and the resource efficiency is high.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of wireless communication networks, in particular to a wireless sensor network clustering routing method and system based on hybrid intelligent optimization. BACKGROUND

[0002] A wireless sensor network is composed of a large number of resource-constrained sensor nodes, and is widely used in environmental monitoring, intelligent agriculture, industrial monitoring and military reconnaissance. However, the sensor nodes are usually powered by batteries, and the energy is extremely limited and difficult to supplement or replace. Therefore, designing an efficient and energy-saving clustering routing protocol to maximize the network lifetime is one of the core challenges of WSN research.

[0003] The classical clustering protocol (such as LEACH) balances energy consumption by randomly selecting cluster heads, but its randomness leads to unstable network topology, uneven energy consumption, and difficulty in adapting to large-scale or heterogeneous networks. In recent years, many intelligent optimization algorithms (such as particle swarm optimization, grey wolf optimizer, etc.) have been introduced to solve the clustering problem, and by constructing a fitness function considering factors such as node energy and location, a better cluster head set can be elected. However, these methods mostly only solve the clustering problem, and do not jointly optimize the routing problem. If a good clustering scheme is not matched by an efficient multi-hop routing, the cluster head may quickly die due to excessive forwarding load, forming an "energy hole" and causing the network to fail prematurely.

[0004] In addition, the WSN environment is dynamic, and the network state (such as node residual energy, link quality) evolves over time. Traditional static optimization methods are difficult to cope with this dynamics, and may lead to the strategy failing soon. Deep reinforcement learning is considered an effective tool for solving dynamic routing problems due to its strong environmental perception and sequential decision-making capabilities. However, DRL has the problem of low training efficiency and slow convergence when solving high-dimensional discrete action space problems (such as cluster head election).

[0005] Therefore, there is no efficient method in the prior art that can combine globally optimized clustering with adaptive dynamic routing to significantly extend the overall network lifetime in a complex and dynamic WSN environment. SUMMARY

[0006] The purpose of the present application is a wireless sensor network clustering routing method and system based on hybrid intelligent optimization, which solves the problem of how to maximize the network lifetime through joint optimization of clustering and routing in a resource-constrained and dynamic network environment.

[0007] To achieve the above purpose, the present application provides the following solutions:

[0008] A wireless sensor network clustering routing method based on hybrid intelligent optimization, comprising:

[0009] constructing a clustering routing model in a wireless sensor network;

[0010] constructing a network energy consumption model based on the clustering routing model, considering the unbalanced energy consumption of nodes;

[0011] constructing a joint optimization problem of cluster head selection and routing path based on the network energy consumption model, aiming to maximize the overall performance of the network;

[0012] transforming the joint optimization problem into a Markov decision process problem according to the dynamic changes of network state;

[0013] solving the Markov decision process problem by using a hybrid intelligent optimization framework combining meta-heuristic and deep reinforcement learning, to obtain the optimal solution of the problem, i.e., the optimal clustering scheme and data transmission path.

[0014] Optionally, establishing a network energy consumption evaluation model comprises:

[0015] modeling the energy consumption of a node as a deterministic function related to communication distance and data volume;

[0016] the network energy consumption evaluation model is:

[0017] ;

[0018] ;

[0019] wherein, E elec is the energy consumption of a transmitting circuit and a receiving circuit for transmitting unit bit data, and are the power amplification coefficients of free space channel and multipath fading channel model respectively, is a distance threshold, is the distance between the transmitter and the receiver, is a deterministic function related to communication distance and data volume, i.e., the energy consumption of a node for transmitting l bit data at a distance of is the size of data (bit number) transmitted or received, is the energy consumption of l bit data at a distance of

[0020] Optionally, constructing a joint optimization problem of cluster head selection and routing path comprises:

[0021] constructing a performance evaluation model based on network lifetime based on the network energy consumption model;

[0022] obtaining multi-objective constraints based on the performance evaluation model and network service priority;

[0023] constructing a joint optimization problem of cluster head selection and routing path based on the multi-objective constraints, with the goal of maximizing network comprehensive performance.

[0024] Optionally, the performance evaluation model is a fitness function of network clustering comprehensive performance:

[0025] The fitness function is:

[0026] ;

[0027] wherein, is a weight coefficient, Res is node residual energy, Dis is the distance between the node and sink, Den is node density, and Load is node load.

[0028] Optionally, the multi-objective constraints include:

[0029] Energy constraint of a single node: ; wherein, is the energy of a single node;

[0030] Network connectivity constraint: ensuring that there is a reachable path between all member nodes in the formed cluster and the cluster head, and between the cluster head and the sink node;

[0031] Cluster size balance constraint:

[0032] ;

[0033] In the formula, f is the cluster size balance, N is the total number of nodes in the current WSN network, L is the side length of the monitoring area, and are the power amplification coefficients of the free space channel and the multipath fading channel model respectively, is the distance between the transmitter and the receiver.

[0034] Optionally, the joint optimization problem is: a joint optimization problem formed by maximizing the fitness function value as the goal and satisfying the multi-objective constraints.

[0035] The joint optimization problem is: maximizing fitness;

[0036] wherein:

[0037] Node residual energy:

[0038] ;

[0039] Wherein, M represents the number of surviving nodes in WSN, f is the optimal cluster head number, E is the node remaining energy;

[0040] Distance between node and sink:

[0041] ;

[0042] Wherein, is the distance between the candidate cluster head and the sink, represents the distance between other cluster nodes in the cluster and the cluster head within the communication range, is the distance between other cluster nodes in the cluster and the sink within the communication range, and n represents the number of nodes in the cluster;

[0043] Node density:

[0044] ;

[0045] Wherein, is the number of nodes within the communication range of the candidate cluster head, represents the distance between the bth and qth sensor nodes;

[0046] Node load:

[0047] ;

[0048] ;

[0049] ;

[0050] Wherein, represents the remaining energy of the cluster head , represents the number of child nodes responsible for by the cluster head , is the average load of all cluster heads, is the total number of cluster heads, is the load of the nth node.

[0051] Optionally, the Markov decision process problem comprises: a state set S, an action space A, a state transition probability P, and a reward function Re;

[0052] Converting the joint optimization problem into a Markov decision process problem comprises:

[0053] Taking the remaining energy and location information of all nodes in the network as the state set S;

[0054] Taking the cluster head election result and the selection of inter-cluster routing path in each round as the action space A; ​

[0055] The reward function Re is:

[0056] ;

[0057] ;

[0058] ;

[0059] wherein, and are the normalized residual energy and distance of the network node respectively, is the current residual energy of node j, is the initial energy, is the current hop count of node j to the sink, is the maximum hop count in the network, denotes the reachability flag of node j, and denote the normalized weight factors of residual energy and node distance respectively.

[0060] Optionally, the hybrid intelligent optimization framework combining meta-heuristic and deep reinforcement learning is an HGWO-AD3QN framework based on an improved grey wolf optimization algorithm and an enhanced double deep Q network; wherein the improved grey wolf optimization algorithm HGWO is: introducing a Tent chaotic mapping to initialize the population in the GWO algorithm, and enhancing the global search ability by improving the Levy flight strategy; the improved grey wolf optimization algorithm HGWO is used to solve the discrete cluster head selection problem, and the enhanced double deep Q network AD3QN is used to solve the continuous routing decision problem.

[0061] Optionally, solving the Markov decision process problem by using the hybrid intelligent optimization framework combining meta-heuristic and deep reinforcement learning comprises:

[0062] S1. Initialize the HGWO algorithm: generate an initial grey wolf population using a Tent chaotic mapping, wherein the position vector of each grey wolf represents a candidate cluster head set;

[0063] S2. Evaluate and update the clustering scheme: calculate the fitness value corresponding to the position of each grey wolf, the fitness value being based on the node residual energy, the node distance from the sink, the node density, and the node load; update the grey wolf position according to the HGWO algorithm, wherein an improved Levy flight strategy is introduced in the global search stage to enhance the population diversity and avoid the algorithm from falling into local optimum too early;

[0064] S3. HGWO outputs the optimal clustering: repeat step S2 until the HGWO algorithm reaches the maximum number of iterations or converges, and output the globally optimal cluster head election scheme;

[0065] S4. Constructing the AD3QN State Space: The energy and geographical location information of each cluster head in the optimal cluster head set obtained in step S3, along with the location of the sink node, are used to construct the state space of the AD3QN network. Based on this state representation, AD3QN further optimizes the network structure by combining a pre-training mechanism, a multi-step Q-learning method, and a Dynamic Feature Fusion (DFF) layer: pre-training is used to improve the model's adaptability to the initial environment, the multi-step method enhances the stability of the estimation of future cumulative rewards, and the DFF layer can dynamically extract and fuse multi-source features such as energy and location information, thereby improving the overall quality of state representation and routing decision performance.

[0066] S5. AD3QN optimizes and outputs complete routing path: The state space is input into the AD3QN network. The AD3QN network calculates the Q value for all possible routing actions in the current network state through forward propagation, and outputs the optimal action with the highest long-term cumulative reward based on its competitive architecture. This sequence of actions constitutes the globally optimal inter-cluster multi-hop routing path from all cluster heads to the sink node, completing the optimization of the entire network data backhaul path.

[0067] S6. Execute a communication round: The network executes a complete data acquisition and transmission round based on the clustering scheme determined in S3 and the routing path determined in S5.

[0068] Optionally, the initial gray wolf population can be generated using the Tent chaotic mapping as follows:

[0069] ;

[0070] in, Represents the current individual's position. , , This represents the optimal position for the three alpha wolves. This represents the chaotic perturbation factor. Represents a chaotic sequence generated by a Tent map;

[0071] Improvements to Levi's flight strategy include:

[0072] An exponential decay factor and a periodic perturbation term are introduced to improve the standard Levy flight, so that the step size changes dynamically with the individual number.

[0073] The optimized Levy step size is expressed as:

[0074] ;

[0075] in, Scaling factor For the current individual index, The improved calculation of Levy's flight stride length, The stride length for the original standard Levi flight. This is an exponentially decaying term, which causes the step size to gradually decrease with each iteration (ensuring stable search in later stages). The parameter used to control the exponential decay and the frequency of periodic oscillations is half the population size.

[0076] The beneficial effects of this invention are as follows:

[0077] This invention combines global clustering optimization with adaptive routing decision-making through joint optimization using the hybrid Grey Wolf optimization algorithm and the AD3QN deep reinforcement learning network. This effectively addresses the energy voids and short network lifetimes caused by the separation of clustering and routing optimization in WSNs. This method leverages the advantages of metaheuristic algorithms in global search and combinatorial optimization while utilizing the intelligence of DRL in sequential decision-making in dynamic environments, significantly improving the overall performance and adaptability of the network. Furthermore, the algorithm boasts fast convergence and high resource efficiency. Attached Figure Description

[0078] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0079] Figure 1 This is a flowchart of a wireless sensor network clustering routing method based on hybrid intelligent optimization, according to an embodiment of the present invention.

[0080] Figure 2 This invention provides a deep learning framework for an enhanced dual-deep Q-network.

[0081] Figure 3 This is a node deployment model for an embodiment of the present invention;

[0082] Figure 4 This is a deep structural framework for an enhanced dual-depth Q-network according to an embodiment of the present invention. Detailed Implementation

[0083] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0084] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0085] like Figure 1 As shown, this embodiment proposes a clustering routing method for wireless sensor networks based on hybrid intelligent optimization, including:

[0086] In wireless sensor networks, a clustered routing model is constructed based on factors such as energy and distance;

[0087] Based on the clustered routing model, and considering the uneven energy consumption of nodes, a network energy consumption model is constructed.

[0088] Based on the network energy consumption model, a performance evaluation model based on network lifetime is constructed.

[0089] Based on the performance evaluation model and network service priorities, obtain multi-objective constraints;

[0090] Based on the aforementioned multi-objective constraints, and with the goal of maximizing the overall network performance, a joint optimization problem of cluster head selection and routing path is constructed.

[0091] Based on the dynamic changes in network state, the joint optimization problem is transformed into a Markov decision process problem;

[0092] A hybrid intelligent optimization framework that integrates metaheuristics and deep reinforcement learning is adopted to solve the Markov decision process problem and obtain the optimal solution, namely the optimal clustering scheme and data transmission path.

[0093] Furthermore, clustering routing models based on factors such as energy and distance include:

[0094] The architecture consists of a large number of randomly deployed heterogeneous sensor nodes, a fixed-location aggregation node, and a hierarchical network structure based on clusters. It is assumed that all nodes have sensing and wireless communication capabilities, and the cluster head node is responsible for the fusion of data within its cluster and multi-hop data forwarding to the aggregation node.

[0095] Furthermore, considering the uneven energy consumption of nodes, the network energy consumption model is constructed as follows:

[0096] The node energy consumption is divided into four parts: sensing, processing, receiving, and transmitting. Transmitting energy consumption is proportional to the nth power of the transmission distance. Its core energy consumption model is as follows:

[0097] ;

[0098]

[0099] In the formula, E elecThe energy consumption for transmitting a unit bit of data in the transmitting and receiving circuits. and These are the power amplification factors for the free-space channel and multipath attenuation channel models, respectively. Distance threshold This represents the distance between the sending and receiving ends.

[0100] Furthermore, constructing a performance evaluation model based on network lifetime includes: constructing a fitness function to evaluate the merits of clustering schemes;

[0101] The fitness function is:

[0102] ;

[0103] In the formula, These are weighting coefficients, Res represents the remaining energy of the node, Dis represents the distance between nodes, Den represents the node density, and Load represents the node load.

[0104] Furthermore, based on the performance evaluation model and the objective of selecting the optimal cluster head, the multi-objective constraints are obtained, including:

[0105] Define the energy constraints for a single node: ;

[0106] Define network connectivity constraints: ensure that there are reachable paths between all member nodes within a cluster and the cluster head, as well as between the cluster head and the sink node;

[0107] Define cluster size balance constraints (optimal number of cluster heads f):

[0108] ;

[0109] In the formula, N is the total number of nodes in the current WSN network, and L is the side length of the monitoring area.

[0110] Furthermore, considering multi-objective constraints, and aiming to maximize overall network performance, a joint optimization problem of cluster head selection and routing path is constructed, including:

[0111] The joint optimization problem is formed by maximizing the fitness function value and satisfying the aforementioned constraints.

[0112] The mathematical formulation of the joint optimization problem is: maximizing fitness;

[0113] C1: Remaining energy at node:

[0114] ;

[0115] In the formula, M represents the number of surviving nodes in the WSN. f is the optimal number of cluster heads. E is the remaining energy of the nodes.

[0116] C2: Distance between node and sink:

[0117] ;

[0118] in, This represents the distance between the candidate cluster head and the sink. This indicates the distance from other nodes within the existing cluster to the cluster head within the communication range. This represents the distance from other nodes within the existing cluster to the sink within the communication range.

[0119] C3: Node density:

[0120] ;

[0121] in, This represents the number of nodes within the communication range of the candidate cluster head. This represents the distance between the b-th and q-th sensor nodes.

[0122] C4: Node Load:

[0123] ;

[0124] ;

[0125] ;

[0126] in, Indicates cluster head The remaining energy. Representative cluster head The number of child nodes it is responsible for (i.e., the number of members within its cluster). It is the average load of all cluster heads, which measures the overall load level of the network. It represents the total number of cluster heads. It is the first The load of each node.

[0127] Furthermore, transforming the joint optimization problem into a Markov decision process problem includes:

[0128] The Markov decision process problem consists of a set of states S, an action space A, state transition probabilities P, and a reward function Re;

[0129] The remaining energy, geographical location information of all cluster head nodes in the network, and the location of the sink node are used as the state set S;

[0130] The cluster head election result in each round is defined as discrete action A1, and the routing path selection from the cluster head to the sink node is defined as discrete action A2, which together constitute the action space A.

[0131] The reward function Re is:

[0132] ;

[0133] ;

[0134] ;

[0135] In the formula, and These are the normalized remaining energy and distance of the network nodes, respectively. Let j be the current remaining energy. This is the initial energy. It is the number of hops from the current node to the sink. It is the maximum number of hops in the network (normalized scale). The reachability flag for node j indicates whether the node is available or has a willingness to forward messages (1 for reachable and 0 for unreachable). and These are the normalized weighting factors representing the remaining energy and node distance, respectively, and satisfy the following conditions: . , The value can be adjusted as needed.

[0136] Furthermore, the hybrid intelligent optimization framework integrating metaheuristics and deep reinforcement learning is: the HGWO-AD3QN framework based on the improved Grey Wolf Optimization algorithm and the enhanced dual deep Q network; wherein, the improved Grey Wolf Optimization algorithm HGWO is: introducing the Tent chaotic mapping to initialize the population in the GWO algorithm, and enhancing the global search capability by improving the Lévy flight strategy; the improved Grey Wolf Optimization algorithm HGWO is used to solve the discrete cluster head selection problem, and the enhanced dual deep Q network AD3QN is used to solve the continuous routing decision problem.

[0137] Furthermore, a hybrid intelligent optimization framework integrating metaheuristics and deep reinforcement learning is employed to solve the Markov decision process problem, including:

[0138] S1. Initialize the HGWO algorithm: Use the Tent chaotic mapping to generate an initial gray wolf population, where the position vector of each gray wolf represents a set of candidate cluster heads;

[0139] S2. Evaluation and update of clustering scheme: Calculate the fitness value corresponding to each gray wolf position, which is based on the node's remaining energy, the distance between the node and the sink, the node density, and the node load; update the gray wolf positions according to the mixed HGWO algorithm, wherein an improved Levy flight strategy is introduced in the global search phase to enhance population diversity and avoid the algorithm from getting trapped in local optima too early.

[0140] S3.HGWO Output Optimal Clustering: Repeat step S2 until the HGWO algorithm reaches the maximum number of iterations or converges, and outputs the globally optimal cluster head election scheme.

[0141] S4. Constructing the AD3QN State Space: The energy and geographical location information of each cluster head in the optimal cluster head set obtained in step S3, along with the location of the sink node, are used to construct the state space of the AD3QN network. Based on this state representation, AD3QN further optimizes the network structure by combining a pre-training mechanism, a multi-step Q-learning method, and a Dynamic Feature Fusion (DFF) layer: pre-training is used to improve the model's adaptability to the initial environment, the multi-step method enhances the stability of the estimation of future cumulative rewards, and the DFF layer can dynamically extract and fuse multi-source features such as energy and location information, thereby improving the overall quality of state representation and routing decision performance.

[0142] S5. AD3QN optimizes and outputs complete routing path: The state space is input into the AD3QN network. The AD3QN network calculates the Q value for all possible routing actions in the current network state through forward propagation, and outputs the optimal action with the highest long-term cumulative reward based on its competitive architecture. This sequence of actions constitutes the globally optimal inter-cluster multi-hop routing path from all cluster heads to the sink node, completing the optimization of the entire network data backhaul path.

[0143] S6. Execute a communication round: The network executes a complete data acquisition and transmission round based on the clustering scheme determined in S3 and the routing path determined in S5.

[0144] In this embodiment, the method for constructing a WSN clustering routing model based on hybrid intelligent optimization by introducing a hierarchical clustering mechanism into the wireless sensor network model includes:

[0145] A hierarchical network architecture is adopted, deploying energy-heterogeneous sensor nodes and fixed-location aggregation nodes within the monitoring area. An improved hybrid gray wolf optimization algorithm is used for cluster head election, comprehensively considering node remaining energy, communication distance, node load, and density. The cluster head node is responsible for data fusion and selects the optimal multi-hop routing path to the aggregation node based on an enhanced dual-deep Q network, achieving efficient and reliable data transmission.

[0146] In this embodiment, the method for building HGWO-AD3QN on the WSN plane deployment model includes:

[0147] The principle of the HGWO algorithm used in the clustering stage:

[0148] GWO is a swarm intelligence optimization algorithm inspired by the hunting behavior of gray wolves. Its core idea is to perform a global optimization search by simulating the social hierarchy and prey-hunting process of gray wolves.

[0149] In the population structure, gray wolves are divided into four ranks: and These correspond to leaders and followers, respectively. The three types of individuals are used to guide the search direction. Individuals update their positions to enhance population diversity. The algorithm mainly comprises three key mechanisms: surrounding prey, chasing prey, and attacking prey. During the iteration process, individuals update their positions based on... The relative position is constantly adjusted to achieve a dynamic balance between global exploration and local development.

[0150] The working principle of GWO is as follows:

[0151] Step 1. The coefficient vectors A and C are guided by random perturbation and distance scaling mechanisms to explore the search space.

[0152] Step 2. The convergence factor 'a' decreases linearly with the number of iterations, thereby enhancing the global search capability in the early stage and strengthening the local fine search in the later stage.

[0153] Step 3. The individual position update formula is shown in the following formula, which realizes the continuous approximation of the optimal region.

[0154] ;

[0155] ;

[0156] ;

[0157] ;

[0158] ;

[0159] In the formula, Represents the current individual's location. , , Represent Current location. , It is a coefficient vector. It is a linearly decreasing vector, decreasing from 2 to 0, controlling exploration and development. , This represents the current iteration number.

[0160] To overcome the local optima problem caused by decreased population diversity in the standard GWO algorithm, the HGWO algorithm introduces a Tent mapping mechanism to enhance the uniformity of the initial population distribution, and combines this with improved levy flight to enhance global escape capability, aiming to balance the algorithm's convergence and global exploration ability. Simultaneously, a fitness function integrating node remaining energy, distance, density, and load factors is designed to achieve energy balance.

[0161] (1) Population initialization based on Tent mapping:

[0162] Tent mappings possess strong ergodicity and initial population diversity generation capabilities, effectively avoiding premature convergence caused by uneven population distribution. A Tent mapping can be represented as:

[0163] ;

[0164] In the formula To control the parameters, a value of 2 is typically used, resulting in a completely chaotic system. The population update formula optimized using Tent mapping is then replaced with the following:

[0165] ;

[0166] In the formula Represents the current individual's location. , , This represents the optimal position for the three alpha wolves. This represents the chaotic perturbation factor. This represents a chaotic sequence generated by a Tent mapping.

[0167] (2) Improved Levi's flight:

[0168] Standard GWO (Global Wandering) is prone to getting trapped in local optima and lacks global exploration capabilities. While standard Lévy flight possesses strong global search capabilities, it also suffers from problems such as difficulty in controlling jump amplitude, lack of directional guidance, weak local search capabilities, and a tendency to premature convergence. Therefore, this embodiment introduces an exponential decay factor and a periodic perturbation term to improve standard Lévy flight, making the step size dynamically change with the individual number, thereby effectively enhancing the accuracy of local searches while maintaining global search capabilities.

[0169] The standard levy step size can be expressed as:

[0170] ;

[0171] in, , The two are independent of each other. This is the levy parameter, which is usually set to 1.5. It is a scale parameter, and its calculation formula can be expressed as:

[0172] ;

[0173] in It is the Gamma function.

[0174] The optimized levy step size can be expressed as:

[0175] ;

[0176] in It is the scaling factor, which is 0.88 in this embodiment. It is the current individual index (the individual ranked first). (Half the population size).

[0177] After adopting the improved Lévy flight optimization, the population renewal formula can be expressed as follows:

[0178] ;

[0179] in Can be calculated The formula is obtained, which represents the current position or current solution of an individual (such as a gray wolf) in the current iteration step. This represents the optimized levy step size.

[0180] The principle of the AD3QN algorithm used in the routing section:

[0181] Step 1. Pre-training. In the reinforcement learning path optimization task of WSN, the agent is usually in a policy-free state in the early stage of training, relying entirely on random exploration. This approach is inefficient and prone to getting stuck in local policies, affecting overall performance. To improve the policy quality of the agent in the early stage of training, this embodiment introduces an expert demonstration mechanism to pre-train the policy network through imitation learning.

[0182] In this embodiment, the pre-training process employs the behavior cloning method, using expert behavior as the target output to optimize the parameters of the neural network. This process does not involve environmental interaction and relies purely on expert trajectories to learn the policy mapping function.

[0183] Step 2. The agent receives the current state s in the environment and inputs it into the neural network. The state feature vector is extracted through two fully connected layers (fc1, fc2).

[0184] Step 3. The traditional D3QN model has significant shortcomings when dealing with WSN path planning problems: the original state coding method lacks modeling of topological information between nodes, making it difficult to characterize the global structural features in the context of multi-hop forwarding.

[0185] To address the aforementioned issues, this embodiment employs a structural enhancement mechanism designed to systematically optimize expressive power and stability at the network level. Specifically, the introduced Dynamic Feature Fusion layer extracts features such as distance and energy from the original state and uses a gating mechanism to dynamically weight and combine different types of information, thereby enhancing the expressive strength of key features and suppressing interference from redundant features.

[0186] ;

[0187] in This represents the input feature vector. , This represents a learnable weight matrix used for linear transformations. , This indicates the bias term. This indicates the "OR" operation in the gating mechanism. This represents the activation function.

[0188] Step 4. The Dueling structure feeds the extracted features into two independent branches: the state value branch and the action advantage function branch.

[0189] 1) State value 𝑉(𝑠): Evaluates the long-term value of state s.

[0190] 2) Action advantage 𝐴(𝑠, 𝑎): Evaluate the advantage of action 𝑎 relative to the average level and output it through the aggregation layer.

[0191] Step 5. Q-value fusion layer (Q-value Output);

[0192] Combine 𝑉(𝑠) and 𝐴(𝑠, 𝑎) to calculate the Q-value, and select the action with the largest Q-value as the optimal action for the current policy. This can be represented as:

[0193] ;

[0194] Q It is the action value of taking action 𝑎 under state 𝑠, which is the Q value of the final output. This represents the average of the dominance function for all actions. Subtracting this term ensures that the dominance function is zero-mean, thereby enhancing numerical stability (preventing excessive shift of the dominance function). It is an action index, and the optimal action is selected by the current network for the next state. These are the parameters of the neural network, including shared parameters and parameters for the two branches (𝑉 and 𝐴).

[0195] Step 6. To alleviate the problem of Q-value overestimation, AD3QN introduces a Double Q-Learning mechanism, which uses the current network and the target network to select and evaluate actions respectively.

[0196] Action selection (by the current network):

[0197] ;

[0198] Motion evaluation (by the target network):

[0199] ;

[0200] Among them, Q It is a target network evaluation, which is based on the target network. Evaluate The value of. This represents the current network parameters, which are the main network parameters that are updated in real time. This represents the target network parameters, which are stable network parameters that are updated with a delay. It is the optimal action selection, where the network selects the optimal action for the next state.

[0201] Step 7. To further enhance the agent's ability to evaluate long-term rewards, this embodiment introduces a multi-step temporal difference (n-step TD) method into the AD3QN framework to construct a more accurate target Q-value. The one-step TD objective used in traditional D3QN only considers the current step reward and the estimated value of the next state; its target Q-value (TD objective) is:

[0202] ;

[0203] in This is used as a supervision signal during training, representing the ideal Q value. This indicates an immediate reward, that is, the direct reward returned by the environment after an action is performed. This represents the discount factor, which is responsible for balancing the importance of current rewards and future rewards (usually set to 0.99).

[0204] In contrast, multi-step TD objectives consider the cumulative reward over n consecutive steps, which helps improve the foresight and stability of value estimation. Its objective Q-value expression is:

[0205] ;

[0206] in, Representing the Instant rewards for each step This represents the state after step n.

[0207] By introducing a multi-step approach, the agent can more effectively propagate long-term benefit signals to the current decision, thereby improving training efficiency.

[0208] Step 8. The loss function measures the difference between the network's output Q-value and the target Q-value. The network optimizes its parameters by minimizing the error between the predicted Q-value and the target Q-value. The loss function is in the form of mean squared error (MSE), and is calculated as follows:

[0209]

[0210] in, It is the value of the loss function. These are the parameters optimized through gradient descent during the parameter vector training process. It is the expectation operator, representing the expected value of the transferred sample sampled from the experience replay buffer 𝐷. It is a scalar quantity, representing an immediate reward. It is a state vector, representing the state transitioned to after the next state performs action 𝑎.

[0211] Step 9. To maintain the stability of the target network, a soft update strategy is used to gradually update the target network parameters. The calculation formula is:

[0212] ;

[0213] in, The soft update coefficient ranges from 0.001 to 0.01.

[0214] exist Figure 2 Within the algorithmic framework shown, the agent continuously interacts with the environment, constantly utilizing the constructed state space, action space, and reward function to gradually learn the optimal path selection strategy. To achieve this goal, this embodiment formalizes the path selection problem as a Markov Decision Process (MDP), which consists of four core elements: agent, state space, action space, and reward function. The specific definitions are as follows:

[0215] 1) Intelligent Agent. In this embodiment, an intelligent agent refers to each cluster head node.

[0216] 2) State Space. This represents the current service type of the mobile terminal (MT). The currently selected network node number n, and the network status parameters of node n. If these factors are also used as state inputs for network selection decisions, then the entire state space S is: ;

[0217] 3) Action Space. This refers to the next hop node that the agent can choose in the current state. The action space is typically a set of neighboring nodes, with each action corresponding to one possible neighboring node. The action space A can be represented as:

[0218] ;

[0219] 4) Reward function. Used to evaluate the quality of actions and guide the agent to choose a better strategy.

[0220] The energy of nodes in a network is finite; therefore, to make the enhancement of the D3QN network more efficient, the following selection methods are needed.

[0221] The optimal path, in this embodiment, uses an exponential function as the reward function, taking into account both the remaining energy of the node and the distance. The calculation method is as follows:

[0222] ;

[0223] ;

[0224] ;

[0225] In the formula, and These are the normalized remaining energy and distance of the network nodes, respectively. Let j be the current remaining energy. This is the initial energy. It is the number of hops from the current node to the sink. It is the maximum number of hops in the network (normalized scale). The reachability flag for node j indicates whether the node is available or has a willingness to forward messages (1 for reachable and 0 for unreachable). and These are the normalized weighting factors representing the remaining energy and node distance, respectively, and satisfy the following conditions: . , The value can be adjusted as needed.

[0226] In this embodiment, the steps for determining the optimal path from the cluster head to the sink are as follows:

[0227] Algorithm 1: AD3QN based routing algorithm Input: initial parameters θ of AD3QN, initial state S_0 (including all W, energy, distance information) Output: optimal routing path from cluster head to sink 1: for 1 to E do: 2: Initialize state . ]] 3: for k=1 to T do: 4: build current state . 5: Use routing policy Select action 6: Perform action , update network environment and observe next state . 7: Calculate rewards 8: Store the experience sample into the priority experience replay buffer; 9: select sampling batch. 10: compute . 11: select target action. 12: compute loss function . 13: update current online network parameters 14: Sync target network parameters: . 15: Update policy . 16: end for 17: reconstruct routing path. 18: end for

[0228] The specific conditions for this example are: 200 nodes, a 100x100 square deployment area, and each node having an initial energy of 0.1 J. The energy consumption of the transmitting and receiving circuits for transmitting one bit of data is also described. 50nJ·bit-1 Power amplification factor of free-space channel and multipath attenuation channel models and 10pJ·bit respectively -1 ·m -2 and 0.0013pJ·bit -1 ·m -4 The data is grouped into 4000 bits each.

[0229] Figure 3 The deployment model of this example was studied, with 200 nodes randomly deployed in a 100x100 square area. The sink was located in the center of the area, possessing sufficient computing power to process all data from the cluster heads. After the cluster head election was completed during the clustering phase, each node joined the nearest cluster to complete the clustering. Nodes within a cluster collected the raw monitoring data and transmitted it to the cluster head node, which was responsible for uploading the fused data to the sink.

[0230] Figure 4 The architecture of AD3QN was studied. First, the input layer receives state information from the environment and extracts features stepwise through two fully connected layers (fc1 and fc2). A dynamic feature fusion module is introduced after each layer to enhance feature representation capabilities. Subsequently, the network is divided into a Value Branch and an Advantage Branch, which calculate the state value 𝑉(𝑠) and the advantage of each action 𝐴(𝑠, 𝑎), respectively. Finally, the two are combined according to a Dueling structure to obtain the Q-value output, providing the agent with a basis for action selection.

[0231] This embodiment also proposes a clustering routing system for wireless sensor networks based on hybrid intelligent optimization, including: a clustering routing model construction module, an energy consumption model construction module, a performance evaluation model construction module, a constraint design module, an optimization problem construction module, a hybrid intelligent optimization algorithm design module, and an optimal solution calculation module;

[0232] The clustering routing model building module is used to build clustering routing models based on factors such as energy and distance in wireless sensor networks.

[0233] The energy consumption model construction module is used to construct a network energy consumption model based on the clustered routing model of this embodiment, taking into account the uneven energy consumption of nodes.

[0234] The performance evaluation model construction module in this embodiment is used to construct a performance evaluation model based on network lifetime according to the network energy consumption model in this embodiment.

[0235] The constraint design module in this embodiment is used to obtain multi-objective constraints based on the performance evaluation model of this embodiment.

[0236] The optimization problem construction module in this embodiment is used to construct a joint optimization problem of cluster head selection and routing path by combining the multi-objective constraints of this embodiment with the goal of maximizing the overall network performance.

[0237] The hybrid intelligent optimization algorithm design module in this embodiment is used to handle the non-convexity and combinatorial complexity of the transformed Markov decision process problem by adopting a hybrid intelligent optimization framework that integrates metaheuristics and deep reinforcement learning.

[0238] The optimal solution calculation module in this embodiment is used to adjust the model parameters of the hybrid optimization framework, train the network model, and obtain the optimal solution to the problem until the model converges.

[0239] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A clustering routing method for wireless sensor networks based on hybrid intelligent optimization, characterized in that, include: Constructing a clustered routing model in wireless sensor networks; Based on the clustered routing model, considering the uneven energy consumption of nodes, a network energy consumption model is constructed. Based on the network energy consumption model, and with the goal of maximizing the overall network efficiency, a joint optimization problem of cluster head selection and routing path is constructed. Based on the dynamic changes in network state, the joint optimization problem is transformed into a Markov decision process problem; A hybrid intelligent optimization framework that integrates metaheuristics and deep reinforcement learning is adopted to solve the Markov decision process problem and obtain the optimal solution, namely, the optimal clustering scheme and data transmission path.

2. The wireless sensor network clustering routing method based on hybrid intelligent optimization according to claim 1, characterized in that, Establishing a network energy consumption assessment model includes: The energy consumption of a node is modeled as a deterministic function related to communication distance and data volume; The network energy consumption assessment model is as follows: ； ； In the formula, E elec The energy consumption for transmitting a unit bit of data in the transmitting and receiving circuits. and These are the power amplification factors for the free-space channel and multipath attenuation channel models, respectively. Distance threshold The distance between the transmitting and receiving ends, A deterministic function relating communication distance and data volume, i.e., the distance at which a node sends l bits of data. The energy consumption is as follows The amount of data sent or received. For l bit data at a distance The energy consumption is reduced.

3. The wireless sensor network clustering routing method based on hybrid intelligent optimization according to claim 1, characterized in that, The joint optimization problem of cluster head selection and routing path construction includes: Based on the network energy consumption model, a performance evaluation model based on network lifetime is constructed. Based on the performance evaluation model and network service priorities, multi-objective constraints are obtained. Based on the aforementioned multi-objective constraints, and with the goal of maximizing the overall network performance, a joint optimization problem of cluster head selection and routing path is constructed.

4. The wireless sensor network clustering routing method based on hybrid intelligent optimization according to claim 3, characterized in that, The performance evaluation model is a fitness function for the overall performance of network clustering: The fitness function is: ； in, is the weighting coefficient, Res is the remaining energy of the node, Dis is the distance between the node and the sink, Den is the node density, and Load is the node load.

5. The wireless sensor network clustering routing method based on hybrid intelligent optimization according to claim 4, characterized in that, The multi-objective constraints include: Energy constraints for a single node: ;in, Energy for a single node; Network connectivity constraints: Ensure that there are reachable paths between all member nodes within a cluster and the cluster head, as well as between the cluster head and the sink node; Cluster size balance constraint: ； In the formula, f represents cluster size balance, N is the total number of nodes in the current WSN network, and L is the side length of the monitoring area. and These are the power amplification factors for the free-space channel and multipath attenuation channel models, respectively. This represents the distance between the sending and receiving ends.

6. The wireless sensor network clustering routing method based on hybrid intelligent optimization according to claim 3, characterized in that, The joint optimization problem is a joint optimization problem formed with the objective of maximizing the fitness function value and satisfying the multi-objective constraints.

7. The wireless sensor network clustering routing method based on hybrid intelligent optimization according to claim 1, characterized in that, The Markov decision process problem includes: a set of states S, an action space A, state transition probabilities P, and a reward function Re; Transforming the joint optimization problem into a Markov decision process problem includes: The remaining energy and location information of all nodes in the network are used as the state set S; The cluster head election results and the selection of inter-cluster routing paths in each round are used as the action space A; The reward function Re is: ； ； ； in, and These represent the normalized remaining energy and distance of the network nodes, respectively. Let j be the current remaining energy. As initial energy, It is the number of hops from the current node to the sink. It is the maximum number of hops in the network. This represents the reachability flag of node j. and These represent the normalized weighting factors for remaining energy and node distance, respectively.

8. The wireless sensor network clustering routing method based on hybrid intelligent optimization according to claim 1, characterized in that, The hybrid intelligent optimization framework integrating metaheuristics and deep reinforcement learning is the HGWO-AD3QN framework, which combines an improved Grey Wolf Optimization (GWO) algorithm with an enhanced dual deep Q-network. The improved GWO algorithm introduces a Tent chaotic mapping to initialize the population and enhances global search capabilities through an improved Lévy flight strategy. The improved GWO algorithm is used to solve discrete cluster head selection problems, while the enhanced dual deep Q-network AD3QN is used to solve continuous routing decision problems.

9. The wireless sensor network clustering routing method based on hybrid intelligent optimization according to claim 8, characterized in that, A hybrid intelligent optimization framework integrating metaheuristics and deep reinforcement learning is used to solve the Markov decision process problem, including: S1. Initialize the HGWO algorithm: Use the Tent chaotic mapping to generate an initial gray wolf population, where the position vector of each gray wolf represents a set of candidate cluster heads; S2. Evaluation and update of clustering scheme: Calculate the fitness value corresponding to each gray wolf position, which is based on the node's remaining energy, the distance between the node and the sink, the node density, and the node load; update the gray wolf positions according to the mixed HGWO algorithm, wherein an improved Levy flight strategy is introduced in the global search phase to enhance population diversity and avoid the algorithm from getting trapped in local optima too early. S3.HGWO Output Optimal Clustering: Repeat step S2 until the HGWO algorithm reaches the maximum number of iterations or converges, and outputs the globally optimal cluster head election scheme. S4. Constructing the AD3QN State Space: The energy and geographic location information of each cluster head in the optimal cluster head set obtained in step S3, along with the location of the sink node, are used to construct the state space of the AD3QN network. Based on this state representation, AD3QN further optimizes the network structure by combining a pre-training mechanism, a multi-step Q-learning method, and a feature extraction and fusion layer. Pre-training is used to improve the model's adaptability to the initial environment, the multi-step method enhances the stability of the estimation of future cumulative rewards, and the feature extraction and fusion layer can dynamically extract and fuse multi-source features such as energy and location information, thereby improving the overall quality of state representation and routing decision performance. S5. AD3QN optimizes and outputs complete routing path: The state space is input into the AD3QN network. The AD3QN network calculates the Q value for all possible routing actions in the current network state through forward propagation, and outputs the optimal action with the highest long-term cumulative reward based on its competitive architecture. This sequence of actions constitutes the globally optimal inter-cluster multi-hop routing path from all cluster heads to the sink node, completing the optimization of the entire network data backhaul path. S6. Execute a communication round: The network executes a complete data acquisition and transmission round based on the clustering scheme determined in S3 and the routing path determined in S5.

10. The wireless sensor network clustering routing method based on hybrid intelligent optimization according to claim 9, characterized in that, The initial gray wolf population generated using the Tent chaotic mapping is represented as follows: ； in, Represents the current individual's position. , , This represents the optimal position for the three alpha wolves. This represents the chaotic perturbation factor. Represents a chaotic sequence generated by a Tent map; Improvements to Levi's flight strategy include: An exponential decay factor and a periodic perturbation term are introduced to improve the standard Levy flight, so that the step size changes dynamically with the individual number. The optimized Levy step size is expressed as: ； in, Scaling factor For the current individual index, The improved calculation of Levy's flight stride length, The stride length for the original standard Levi flight. The term is an exponentially decaying term, causing the step size to gradually decrease with each iteration. Parameters used to control exponential decay and the frequency of periodic oscillations.

Images