Link-Aware Underwater Routing Protocol Design Method

Abstract

This invention discloses a link-aware underwater routing protocol design method, belonging to the field of underwater wireless sensor communication networking technology. The invention addresses the problem that existing underwater routing protocols cannot guarantee reliable data transmission due to link quality issues. It includes dividing underwater wireless sensors into clusters based on the roles defined in the node control data packets; each cluster includes a master node and corresponding slave nodes; each master node receives sensor data packets from its corresponding slave node; each master node maintains the attribute information of neighboring master nodes based on their master node attribute data packets, and establishes a reward function for transmitting master node data packets between two master nodes based on the master node attribute information; after determining the next-hop master node for underwater routing based on the reward function value, the current master node's data packet is transmitted to the next-hop master node, until all master nodes' data packets are transmitted to the surface vessel receiving terminal. This invention is used for the transmission of underwater sensor data.

Description

Technical Field

[0001] This invention relates to a link-aware underwater routing protocol design method, belonging to the field of underwater wireless sensor communication networking technology. Background Technology

[0002] Underwater wireless sensor networks (UWSNs) are crucial communication infrastructures supporting underwater monitoring applications. However, the inherent high bit error rate, significant transmission delay, and diverse interference factors of underwater acoustic channels, coupled with the high energy consumption and power supply challenges faced by network nodes, all increase the necessity and urgency of designing efficient routing protocols.

[0003] In recent years, with the rapid development of artificial intelligence technology, reinforcement learning, especially Q-learning algorithms, has become an important tool for solving complex decision-making problems. This algorithm, with its adaptive and trial-and-error learning mechanism, allows agents to dynamically adjust their action strategies based on real-time environmental feedback, without relying on extensive prior knowledge, and has been introduced to solve the challenges of underwater wireless sensor routing protocols. However, most existing underwater routing protocols incorporating Q-learning are designed in idealized environments, primarily focusing on optimizing energy consumption and reducing end-to-end latency, often neglecting the significant impact of link quality on data transmission success rates. Due to the inherently high bit error rate and frequent interference in the underwater environment, frequent packet loss in some application scenarios with extremely high data transmission reliability requirements may lead to the need for large amounts of data retransmission, which not only increases energy consumption but also makes it difficult to meet the high reliability requirements of applications. Therefore, developing a robust, reliable, and energy-efficient underwater wireless sensor network routing protocol has become a critical problem that urgently needs to be solved. Summary of the Invention

[0004] To address the problem that existing underwater routing protocols cannot guarantee reliable data transmission due to link quality issues, this invention provides a link-aware underwater routing protocol design method.

[0005] The present invention provides a link-aware underwater routing protocol design method, comprising:

[0006] Deploy an underwater wireless sensor network and divide the underwater wireless sensors into clusters based on the roles defined in the node control data packets; each cluster includes a master node and corresponding slave nodes.

[0007] Each master node acts as an independent intelligent agent, receiving sensor data packets from corresponding slave nodes. Each master node's routing and forwarding data packets include master node attribute data packets and master node sensor data packets. Each master node integrates all received slave node sensor data packets and the current master node sensor data packet into a master node data packet to be transmitted. Simultaneously, each master node maintains the attribute information of neighboring master nodes based on their master node attribute data packets and establishes a reward function for transmitting master node data packets between two master nodes based on their attribute information. After determining the next-hop master node for underwater routing based on the reward function value, the current master node data packet to be transmitted is transmitted to the next-hop master node, until all master node data packets to be transmitted are transmitted to the surface vessel receiving terminal.

[0008] According to the link-aware underwater routing protocol design method of the present invention, in each cluster, the master node and the slave node complete the cluster division through the interaction of node control data packets; wherein the master node sends cluster invitation information through broadcast, and the slave node sends an acknowledgment reply information to the master node after receiving the invitation information, thereby realizing the establishment of the cluster.

[0009] According to the link-aware underwater routing protocol design method of the present invention, the attribute information in the master node attribute data packet includes the current remaining energy E of the master node. pre Initial energy E of the master node ini Average energy of all adjacent master nodes The depth difference D between the master node and the surface ship k The maximum depth difference D between the master node and the surface ship max,k And the transmission distance d, where k represents the diffusion coefficient.

[0010] According to the link-aware underwater routing protocol design method of the present invention, the reward function is established as follows: This indicates that the master node i will transmit the data packet a to be transmitted by the master node. i The direct reward obtained from the adjacent master node j:

[0011]

[0012] In the formula, χ1 is the energy weighting factor, χ2 is the depth weighting factor, χ3 is the link quality weighting factor, E is the energy factor, D is the depth factor, and J is the link quality factor.

[0013]

[0014] In the formula, SNR is the average signal-to-noise ratio, which is used when the transmission distance from master node i to the adjacent master node j is d:

[0015]

[0016] In the formula E b Let A(d,f) represent the average energy consumed in transmitting one bit of data, where A(d,f) is the path loss, f is the transmission frequency, and α(f) is the absorption coefficient. A(d,f) = d k α(f) d N0 is the noise power spectral density under additive white Gaussian noise conditions.

[0017] According to the link-aware underwater routing protocol design method of the present invention,

[0018] According to the link-aware underwater routing protocol design method of the present invention, assuming there are N master nodes and each master node i has N-1 neighboring master nodes, the reward function is used. Construct an N×N reward matrix R(i) corresponding to master node i:

[0019]

[0020] Let Q represent the value of the reward matrix R(i), then Q K+1 (i,a j () represents the (i,j)th element of the reward matrix R(i) in the (K+1)th iteration, where j = 1, 2, 3, ..., N, and i = 1, 2, 3, ..., N;

[0021]

[0022] In the formula, ξ∈(0,1] is the learning rate, γ∈(0,1] is the discount factor, and v K (j) represents the maximum value in the j-th row of the reward matrix R(i) in the K-th iteration. Q is chosen. K+1 (i,a j The adjacent master node corresponding to the maximum value is taken as the next hop master node.

[0023] According to the link-aware underwater routing protocol design method of the present invention,

[0024] In the formula Let represent the probability of successfully traversing from master node i to its adjacent master node j, and set it to 1. This represents the probability of failure from master node i to the adjacent master node j, and is set to 0. Data packet a to be transmitted by the master node i The reward function that is not propagated outward from the master node i.

[0025] According to the link-aware underwater routing protocol design method of the present invention, the information in the master node attribute data packet includes:

[0026] Source node address: Represents the source node ID of the current master node;

[0027] Destination address: Indicates the address of the surface vessel;

[0028] Next-hop master node address;

[0029] Reward function value: Records the Q of all elements in the reward matrix R(i). K+1 (i,a j ) value, and select the largest Q K+1 (i,a j )value;

[0030] Including the master node's initial energy, the master node's current remaining energy, the master node's location information, and the master node's sensor data packet ID.

[0031] According to the link-aware underwater routing protocol design method of the present invention, the fields included in the node control data packet are:

[0032] Identifier: Used to identify the type of node control data packets;

[0033] Node ID: Information used to identify the node's identity;

[0034] Data: Control nodes control the instructions or control information carried in data packets, and coordinate the clustering and roles of nodes.

[0035] According to the link-aware underwater routing protocol design method of the present invention, the Q value of the reward matrix R(i) is calculated based on the Q-learning algorithm.

[0036] The beneficial effects of this invention are as follows: In the network startup phase, the method of this invention divides nodes into master nodes and slave nodes according to preset roles to optimize communication efficiency and reduce packet collisions. After the nodes are deployed underwater, they automatically identify and control information in the data packets to complete the network role configuration. The master node is responsible for inviting slave nodes to join the cluster and completing the cluster formation. Then, in the communication phase, slave nodes send the collected data to the master node, which then transmits the data to the surface vessel. Due to the inherently high bit error rate and frequent interference in the underwater environment, coupled with the long distance between the master node and the surface vessel, to ensure effective data packet transmission, this invention constructs a reward function equipped with a Q-learning algorithm to build the transmission path. This comprehensively considers the link quality, remaining energy, and depth differences between nodes, improving the rationality and adaptability of selecting the next-hop node. Furthermore, it designs the routing and forwarding data packet format and the control data packet format.

[0037] This invention significantly improves packet delivery rate, demonstrating a 28% performance improvement compared to the SEP protocol and nearly double that of the LEACH protocol. This invention better meets the reliability requirements of underwater applications for network data transmission, achieves efficient packet forwarding, and effectively improves energy efficiency. Attached Figure Description

[0038] Figure 1 This is a flowchart of the link-aware underwater routing protocol design method described in this invention;

[0039] Figure 2 This is a diagram illustrating the interaction between an intelligent agent and its environment.

[0040] Figure 3 This is a schematic diagram of the routing and forwarding data packet format of the master node;

[0041] Figure 4 It is a schematic diagram of the node control data packet format;

[0042] Figure 5 This is a comparison chart of the number of surviving master nodes for different protocol transmission methods in the embodiments;

[0043] Figure 6 is based on Figure 5 Comparison chart of primary node mortality rates;

[0044] Figure 7 This is a comparison chart of the amount of data received by surface ships using different transmission protocols. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0047] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of the invention.

[0048] Specific Implementation Method 1: Combination Figure 1 As shown, this invention provides a link-aware underwater routing protocol design method, including:

[0049] Deploy an underwater wireless sensor network and divide the underwater wireless sensors into clusters based on the roles defined in the node control data packets; each cluster includes a master node and corresponding slave nodes.

[0050] Each master node acts as an independent intelligent agent, receiving sensor data packets from corresponding slave nodes. Each master node's routing and forwarding data packets include master node attribute data packets and master node sensor data packets. Each master node integrates all received slave node sensor data packets and the current master node sensor data packet into a master node data packet to be transmitted. Simultaneously, each master node maintains the attribute information of neighboring master nodes based on their master node attribute data packets and establishes a reward function for transmitting master node data packets between two master nodes based on their attribute information. After determining the next-hop master node for underwater routing based on the reward function value, the current master node data packet to be transmitted is transmitted to the next-hop master node, until all master node data packets to be transmitted are transmitted to the surface vessel receiving terminal.

[0051] As an example, in the initial deployment phase of the network, each node is assigned as a master or slave node according to a pre-defined role policy, and this role information is encapsulated in control packets. These nodes are then deployed to an underwater environment. After activation, they automatically identify the data in the control packets (where 0 represents a master node and 1 represents a slave node) to configure their network roles. Master and slave nodes complete cluster partitioning through the interaction of node control packets. Once roles are configured, in each cluster, the master node initiates the cluster formation process by broadcasting cluster invitation information. The invitation information is received by all slave nodes within the master node's communication range. After receiving the invitation information, the slave nodes send acknowledgment replies to the master node. When the master node receives acknowledgment replies from all slave nodes, the clustering phase of the network is officially completed, thus realizing the establishment of clusters.

[0052] Combination Figure 2 As shown, during network operation, the agent continuously maintains relevant information about neighboring nodes by parsing the received route-forwarded data packets. The attribute information in the master node attribute data packet includes the master node's current remaining energy E. pre Initial energy E of the master node ini The average energy E of all adjacent master nodes, and the depth difference D between the master node and the surface ship. k The maximum depth difference D between the master node and the surface ship max,k And the transmission distance d, where k represents the diffusion coefficient.

[0053] In this embodiment, after selecting a suitable hydrological area, underwater sensor nodes are precisely deployed to ensure optimal node location and network coverage, thereby completing the deployment of the underwater wireless sensor network. After node deployment, the system automatically generates control data packets. Through the interaction mechanism of these control data packets, network nodes automatically complete cluster division according to preset rules and parameters. After cluster formation, each master node is treated as an independent agent during the communication phase. Each agent designs a corresponding reward function based on its interaction with the surrounding environment and dynamically adjusts its action strategy through real-time environmental feedback, thereby continuously optimizing the data transmission path. The reward function is designed by comprehensively considering key factors such as link quality between nodes, remaining energy of nodes, and depth differences to ensure the rationality and efficiency of transmission decisions. To avoid data conflicts during routing, this embodiment explicitly defines the format of routing forwarding data packets and introduces control data packets. In the initial stage of network construction, control data packets are used to coordinate the cluster formation process.

[0054] Furthermore, the design of the reward function is crucial to the performance of the routing protocol. In this implementation, link quality, remaining energy, and depth differences are incorporated into the design of the reward function; the reward function is established based on the information in the master node attribute data packet. This indicates that the master node i will transmit the data packet a to be transmitted by the master node. i The direct reward obtained from the adjacent master node j:

[0055]

[0056] In the formula, χ1 is the energy weighting factor, χ2 is the depth weighting factor, χ3 is the link quality weighting factor, E is the energy factor, D is the depth factor, and J is the link quality factor.

[0057]

[0058] In the formula, SNR is the average signal-to-noise ratio (SNR), which is defined when the transmission distance from master node i to the adjacent master node j is d:

[0059]

[0060] In this embodiment, the Thorp model is used to describe underwater sound wave propagation;

[0061] In the formula E b Let A(d,f) represent the average energy consumed in transmitting one bit of data, where A(d,f) is the path loss, f is the transmission frequency, and α(f) is the absorption coefficient. A(d,f) = d k α(f) d N0 is the noise power spectral density under additive white Gaussian noise (AWGN) conditions.

[0062] In this embodiment,

[0063] Furthermore, assuming there are N master nodes in total, and each master node i has N-1 neighboring master nodes, then the reward function... Construct an N×N reward matrix R(i) corresponding to master node i:

[0064]

[0065] Let Q represent the value of the reward matrix R(i), then Q K+1 (i,a j () represents the (i,j)th element of the reward matrix R(i) in the (K+1)th iteration, where j = 1, 2, 3, ..., N, and i = 1, 2, 3, ..., N;

[0066] The elements of the Q-value in the reward matrix R(i) are updated using the following iterative formula:

[0067]

[0068] In the formula, ξ∈(0,1] is the learning rate, which affects the update speed of the Q matrix; γ∈(0,1] is the discount factor, meaning that the current action will have a greater impact on the current value than future actions; v K (j) represents the maximum value of the j-th row in the K-th iteration of the reward matrix R(i). To ensure that the master node i receives the maximum reward when selecting the next hop node, a propagation path that maximizes the Q-value will be adopted. After iterating through the neighboring master nodes, Q will be selected. K+1 (i,a j The adjacent master node corresponding to the maximum value is taken as the next hop master node.

[0069] The Q-values ​​of each sensor node are updated using a distributed computing method. When the number of data packets locally cached at the source node reaches a certain amount, the optimal next-hop routing node is determined based on the current Q-value.

[0070]

[0071] In the formula Let represent the probability of successfully traversing from master node i to its adjacent master node j, and set it to 1. This represents the probability of failure from master node i to the adjacent master node j, and is set to 0. Data packet a to be transmitted by the master node i The reward function that is not propagated outward from the master node i.

[0072] The transition probability matrix P(i) of N×N can be defined as:

[0073]

[0074] Combination Figure 3 As shown, the information in the master node attribute data packet includes:

[0075] Source node address: Represents the source node ID of the current master node;

[0076] Destination address: The address of the target node to which the data packet should be sent, i.e., the address of the surface ship;

[0077] Next-hop master node address: The address of the node selected as the next hop during the transmission of the data packet;

[0078] Reward function value: The current node calculates the Q-values ​​of its neighboring nodes and records the Q-values ​​of all elements in the reward matrix R(i). K+1 (i,a j ) value, and select the largest Q K+1 (i,a j The value becomes the next hop;

[0079] Initial energy of the master node: The initial energy of the neighboring master nodes, which is helpful for measuring the energy consumption of the master node;

[0080] Master node's current remaining energy: The remaining energy of the current neighboring master nodes, used to assess the master node's ability to continue working;

[0081] Master node location information: The geographical location information of the current neighboring master nodes, used to determine the master node's position in the network and its relative position with other master nodes;

[0082] Master node sensor data packet ID: A unique identifier for each data packet, ensuring the uniqueness of data packets in the network and avoiding duplicate transmission;

[0083] The master node sensor data packet includes data, that is, the actual data content to be transmitted.

[0084] The master node that is currently forwarding data will read Figure 3 The data is used to construct a reward function. The neighbor node with the largest Q-value is selected as the next-hop node. The current master node then forwards the data packet to the node with the largest Q-value and updates the data packet based on the relevant information of the next-hop node. If a suitable next-hop node cannot be selected, the current master node will only update local information and discard the data packet to avoid unnecessary energy consumption and data conflicts.

[0085] Combination Figure 4 As shown, control packets transmit control information to ensure that the network can correctly configure node roles and communication structures during startup, avoiding conflicts during cluster construction. The fields included in the node control packets are:

[0086] Identifier: Used to identify the type of node control data packet, determining whether it is a control data packet or a data packet;

[0087] Node ID: Information used to identify the node's identity; the node ID that sends control data packets helps the receiver identify the sender's identity;

[0088] Data: Control nodes carry instructions or control information in data packets, coordinate node clustering and role allocation, or other network operations.

[0089] In this embodiment, the Q value of the reward matrix R(i) is calculated based on the Q-learning algorithm.

[0090] Example:

[0091] Within a detection area of ​​5000m × 5000m × 5000m, 100 master nodes are evenly distributed. The coordinates of the surface vessel are (2500, 2500, 5000), and the operating frequency is set to 3kHz. Underwater routing is performed using the method of this invention, the LEACH protocol, and the SEP protocol, respectively, to obtain... Figure 5 and Figure 6 The master node survival and death data are shown. It can be seen that the lifecycles of the method of this invention, the LEACH protocol, and the SEP protocol are 189 rounds, 74 rounds, and 123 rounds, respectively. Figure 6 It presents the information more intuitively. Figure 5 The data shows that the protocol proposed in this invention performs excellently from the initial stage. With continuous optimization of the Q-learning algorithm, even in the event of a large number of master node failures in the later stages, this protocol maintains a more stable and continuous lifecycle for master nodes by fully considering link quality. Clearly, the protocol proposed in this invention significantly outperforms the other two protocols in terms of lifecycle, demonstrating higher energy efficiency.

[0092] Figure 7 Furthermore, the proposed protocol, along with LEACH and SEP protocols, transmitted 1949, 1083, and 1521 data packets to the aggregation master node, respectively, indicating that the proposed protocol significantly outperforms other protocols in terms of data transmission volume. Benefiting from a longer lifespan, the proposed protocol transmits almost twice the amount of data as LEACH and achieves approximately 28% better transmission performance than the SEP protocol. As the number of master nodes gradually decreases during protocol operation, the link quality of the channel becomes a critical factor determining the successful forwarding of data packets. By comprehensively considering the impact of link quality on data transmission, the proposed protocol maintains stable data transmission even with a decrease in master nodes in the later stages, achieving efficient and reliable packet forwarding.

[0093] In summary, this invention fully considers the inherent high bit error rate and frequent interference characteristics of the underwater environment, incorporating link quality into the reward function design of the Q-learning algorithm, thereby improving the reliability and delivery success rate of data transmission. This invention designs a control packet format to ensure that nodes can automatically configure roles and communication structures during network startup, effectively avoiding communication conflicts that may occur during cluster construction and improving network initialization efficiency. This invention treats the master node as an intelligent agent and designs a dedicated routing and forwarding packet format. Through the initial energy, remaining energy, and location information of neighboring nodes carried in the packets, the agent can interact with the environment, accelerating the determination of routing paths and optimizing the data transmission process.

[0094] While the invention has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the invention. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that different dependent claims and features described herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other described embodiments.

Claims

1. A link-aware underwater routing protocol design method, characterized in that, include: Deploy an underwater wireless sensor network and group the underwater wireless sensors into clusters based on the roles defined in the node control data packets; Each cluster includes a master node and corresponding slave nodes; Each master node is treated as an independent intelligent agent, receiving sensor data packets from the corresponding slave nodes. Each master node's routing and forwarding data packets include master node attribute data packets and master node sensor data packets. Each master node integrates all received slave node sensor data packets and the current master node sensor data packet into a master node data packet to be transmitted. Simultaneously, each master node maintains the attribute information of adjacent master nodes based on their master node attribute data packets and establishes a reward function for transmitting master node data packets between two master nodes based on the master node attribute information. After determining the next-hop master node for underwater routing based on the reward function value, the current master node data packet to be transmitted is transmitted to the next-hop master node until all master node data packets to be transmitted are transmitted to the surface vessel receiving terminal. The attribute information in the master node attribute data packet includes the master node's current remaining energy E. pre Initial energy E of the master node ini Average energy of all adjacent master nodes The depth difference D between the master node and the surface ship k The maximum depth difference D between the master node and the surface ship max,k And the transmission distance d, where k represents the diffusion coefficient; Establish the reward function as follows This indicates that the master node i will transmit the data packet a to be transmitted by the master node. i The direct reward obtained from the adjacent master node j: In the formula, χ1 is the energy weighting factor, χ2 is the depth weighting factor, χ3 is the link quality weighting factor, E is the energy factor, D is the depth factor, and J is the link quality factor. In the formula, SNR is the average signal-to-noise ratio, which is used when the transmission distance from master node i to the adjacent master node j is d: In the formula E b Let A(d,f) represent the average energy consumed in transmitting one bit of data, where A(d,f) is the path loss, f is the transmission frequency, and α(f) is the absorption coefficient. A(d,f) = d k α(f) d N0 is the noise power spectral density under additive white Gaussian noise conditions.

2. The underwater routing protocol design method based on link awareness according to claim 1, characterized in that, In each cluster, the master node and slave nodes complete the cluster division through the interaction of node control data packets; the master node sends cluster invitation information through broadcast, and the slave nodes send confirmation reply information to the master node after receiving the invitation information, thereby realizing the establishment of the cluster.

3. The underwater routing protocol design method based on link awareness according to claim 2, characterized in that, 4. The underwater routing protocol design method based on link awareness according to claim 3, characterized in that, Assuming there are N master nodes, and each master node i has N-1 neighboring master nodes, then the reward function... Construct an N×N reward matrix R(i) corresponding to master node i: Let Q represent the value of the reward matrix R(i), then Q K+1 (i,a j () represents the (i,j)th element of the reward matrix R(i) in the (K+1)th iteration, where j = 1, 2, 3, ..., N, and i = 1, 2, 3, ..., N; In the formula, ξ∈(0,1] is the learning rate, γ∈(0,1] is the discount factor, and v K (j) represents the maximum value in the j-th row of the reward matrix R(i) in the K-th iteration. Q is chosen. K+1 (i,a j The adjacent master node corresponding to the maximum value is taken as the next hop master node.

5. The underwater routing protocol design method based on link awareness according to claim 4, characterized in that, In the formula Let represent the probability of successfully traversing from master node i to its adjacent master node j, and set it to 1. This represents the probability of failure from master node i to the adjacent master node j, and is set to 0. Data packet a to be transmitted by the master node i The reward function that is not propagated outward from the master node i.

6. The underwater routing protocol design method based on link awareness according to claim 1, characterized in that, The information in the master node attribute data packet includes: Source node address: Represents the source node ID of the current master node; Destination address: Indicates the address of the surface vessel; Next-hop master node address; Reward function value: Records the Q of all elements in the reward matrix R(i). K+1 (i,a j ) value, and select the largest Q K+1 (i,a j )value; Including the master node's initial energy, the master node's current remaining energy, the master node's location information, and the master node's sensor data packet ID.

7. The underwater routing protocol design method based on link awareness according to claim 1, characterized in that, The nodes control data packets contain the following fields: Identifier: Used to identify the type of node control data packets; Node ID: Information used to identify the node's identity; Data: Control nodes control the instructions or control information carried in data packets, and coordinate the clustering and roles of nodes.

8. The underwater routing protocol design method based on link awareness according to claim 4, characterized in that, The Q-value of the reward matrix R(i) is calculated based on the Q-learning algorithm.

Images