Trusted routing protocol in underwater acoustic sensor network

Abstract

The invention relates to a trusted routing protocol in an underwater acoustic sensor network, which comprises the following steps of: decomposing the trusted routing protocol in a UASN into a plurality of sub-problems, and representing the sub-problems as a mathematical problem for maximizing average link quality under a multi-constraint condition; then, multi-dimensional trust features in the UASN are comprehensively considered to calculate trust values of sensing nodes in the UASN; and in combination with a node trust value, designing an invalid action mask method, and based on a deep reinforcement learning algorithm, realizing determination of a next-hop routing relay node, and performing iteration in sequence until a trusted routing path from a source node to a target node is determined, thereby completing reliable transmission of underwater data. The reliability of the candidate routing relay node can be effectively evaluated, the reliability of a routing path is improved, and the security of UASN data forwarding is guaranteed.

Description

Technical Field

[0001] This invention belongs to the field of wireless sensor network technology, specifically relating to a trusted routing protocol in an underwater acoustic sensor network. Background Technology

[0002] Underwater acoustic sensor networks (UASNs) provide strong support for data acquisition, processing, and transmission in marine applications. However, the complexity and uniqueness of the underwater environment pose challenges to underwater data transmission. In UASNs, nodes are typically composed of anchored buoys, autonomous underwater vehicles (AUVs), or bottom sensors, and are limited by underwater deployment conditions and energy supply, making it difficult to establish stable and continuous communication links between nodes. Therefore, reliable routing protocol design is crucial for underwater data transmission.

[0003] Routing protocols in UASNs face unique challenges. First, the characteristics of underwater acoustic channels severely limit the effectiveness of routing decisions: underwater acoustic propagation is slow, bandwidth is extremely narrow, path loss is high, and it is accompanied by significant multipath effects and time-varying noise interference, resulting in highly unstable link quality, making traditional routing strategies based on static link quality assessment difficult to apply. Second, the network topology is highly dynamic: influenced by ocean dynamics factors such as ocean currents, tides, and thermoclines, node positions continuously drift, causing the UASN topology to constantly change, with intermittent connectivity between adjacent nodes, increasing the uncertainty of data transmission and requiring routing protocols to have rapid adaptation and reconstruction capabilities.

[0004] Existing routing protocol research typically transforms the routing problem into an optimization problem with different objectives, such as path shortestization or minimum energy consumption, generally assuming a completely trustworthy network environment while neglecting the trustworthiness of the routing paths. Since UASNs are usually deployed in open, unattended waters with extremely weak physical protection, once nodes are captured or their firmware is tampered with, they can easily become internal attack nodes. Internal attack nodes can infiltrate the network and disrupt normal data transmission under a legitimate identity, posing a serious threat to the security of UASN routing. Therefore, in the complex and dynamic underwater environment, there is an urgent need for a secure routing protocol that integrates node trust assessment and adaptive routing decision-making mechanisms to effectively resist internal attacks while ensuring transmission efficiency and guaranteeing the integrity and reachability of critical data. Summary of the Invention

[0005] The present invention aims to overcome the above-mentioned shortcomings of the prior art and proposes a trusted routing protocol in underwater acoustic sensor networks.

[0006] The concept of this invention is as follows: First, the trusted routing problem in UASN is decomposed into multiple sub-problems for determining trusted routing relays, and expressed as a mathematical problem of maximizing average link quality under multiple constraints; then, the trust value of the sensor nodes in UASN is calculated by comprehensively considering the multi-dimensional trust features in UASN; combined with the node trust value, the next-hop routing relay node is determined based on the deep reinforcement learning (DRL) algorithm, and this process is iterated until a trusted routing path from the source node to the target node is determined, thus completing the reliable transmission of underwater data.

[0007] The technical solution of this invention: A trusted routing protocol in an underwater acoustic sensor network includes the following steps: (1) The trusted routing problem of underwater acoustic sensor network (UASN) is decomposed into multiple single-hop trusted routing relay determination subproblems, and further transformed into an optimization model for optimizing average link quality under multiple constraints. (1.1) Represent UASN as a directed graph, where the set of vertices represents the set of sensor nodes in the network, the set of edges represents the directed communication links between nodes, and define three types of key node roles: source node is the sensor node that is currently forwarding data, relay node is the next hop node for data forwarding, and target node.

[0008] (1.2) The UASN trusted routing problem is decomposed into an iterative single-hop trusted routing relay determination problem, that is, starting from the source node, the forwarding link is established by repeatedly selecting the best next-hop node from the candidate neighbors until the data is forwarded to the target node.

[0009] (1.3) The objective function is to maximize the average link quality in the UASN, where the link quality is the ratio of the link quality from the source node to the relay node to the total number of network nodes.

[0010] (2) The trust value of the sensing node in the UASN is calculated by comprehensively considering the multidimensional trust features in the UASN. The trust value is used to represent the reliability of the node. The node trust evaluation comprehensively considers four dimensions: data, communication, energy, and environment. Addressing the sparsity and deviation of underwater anomalous nodes, the isolated forest algorithm is employed as the core evaluation mechanism. Isolation trees are constructed through recursive random partitioning. Leveraging the statistical characteristics of anomalous nodes being easily isolated and having short paths, the average path length in the isolated forest is inversely mapped to a trust value, thereby achieving accurate identification and trust quantification of hidden anomalous nodes.

[0011] (3) A deep reinforcement learning (DRL) algorithm is used to implement trusted relay decision-making, and an invalid action masking strategy is used to eliminate potentially unreliable nodes. The algorithm runs iteratively until a routing path from the source node to the target node is determined.

[0012] (3.1) Using the DRL algorithm, through its interaction and iteration with the underwater environment, it learns and optimizes its strategy autonomously under the condition of frequent topology changes, overcomes environmental uncertainty, and realizes the decision on the optimal trusted route.

[0013] (3.2) The main elements of the DRL algorithm include the action space, state space, and reward function. The action space at time t... a t The state space contains all neighboring nodes of the current node. s t This includes the source node location, the destination node location, and the trust values ​​of all nodes in the UASN. The reward function has three cases: if the node directly delivers the data to the destination, the system gives the highest positive reward; if it only successfully forwards the data to the relay node, while rewarding high link quality, it will penalize behaviors that generate too many hops or cause the location to move far away from the destination, thus balancing transmission efficiency; and if no acknowledgment message is received, the system will directly give a negative penalty, thereby guiding the route to avoid unreliable paths.

[0014] (3.4) Introduce an invalid action mask to filter abnormal nodes. Based on the trust value obtained in (2), actions with a trust value greater than the threshold are considered valid, and those with a trust value less than the threshold are considered invalid.

[0015] (3.5) Determine the trusted relay node for the next hop based on the DRL algorithm. This process is iterated until the complete routing path from the source node to the destination node is determined.

[0016] The innovation of this invention is: This invention provides a trusted routing protocol for UASNs, deeply integrating node trust mechanisms into the routing decision-making process to effectively address potential internal security threats in open waters. By fusing four-dimensional features of data, communication, energy, and environment, and employing the Isolation Forest algorithm for high-precision identification of sparse and concealed abnormal nodes, dynamic trust assessment is achieved. An adaptive routing framework based on deep reinforcement learning is constructed, decomposing end-to-end trusted path selection into a single-hop relay optimization problem under multiple constraints. An invalid action masking mechanism based on trust thresholds filters highly trusted next-hop nodes, enabling secure, efficient, and reliable data transmission in highly dynamic and unstable underwater environments. This solution overcomes the limitations of traditional routing protocols that neglect security and trustworthiness and are ill-suited to complex marine environments, providing UASNs with both security and robustness guarantees for critical applications such as marine monitoring and underwater security.

[0017] The beneficial effects of this invention are: This invention proposes a trusted routing protocol for underwater acoustic sensor networks (UASNs). The trusted routing protocol in UASNs is represented as a mathematical problem of maximizing average link quality under multiple constraints. Then, the trust value of sensor nodes in the UASN is calculated by comprehensively considering the multi-dimensional trust characteristics. Combining the node trust values, a deep reinforcement learning algorithm is used to determine the next-hop routing relay node. This process is iterated until a trusted routing path from the source node to the target node is determined, completing the reliable transmission of underwater data. This protocol improves the reliability, environmental adaptability, and anti-interference robustness of underwater routing, providing a new technical path for building a highly reliable and adaptive UASN communication system. Attached Figure Description

[0018] Figure 1 This is a flowchart of one embodiment of the present invention; Figure 2 This is a schematic diagram of a UASN attack according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a trusted routing decision based on DRL according to an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0020] The application principle of the present invention will be described in detail below with reference to the accompanying drawings.

[0021] like Figure 1 The diagram illustrates a trusted routing protocol flowchart in an underwater acoustic sensor network (UASN). This invention decomposes the trusted routing protocol in an UASN into multiple sub-problems, representing them as a mathematical problem of maximizing average link quality under multiple constraints. Then, it comprehensively considers the multi-dimensional trust features in the UASN to calculate the trust value of sensor nodes. Combining the node trust values, an invalid action masking method is designed, and a deep reinforcement learning algorithm is used to determine the next-hop routing relay node. This process iterates until a trusted routing path from the source node to the target node is determined, completing the reliable transmission of underwater data. This protocol improves the reliability and robustness of underwater routing. The specific steps of the trusted routing protocol flowchart in an underwater acoustic sensor network are as follows: Step (1): Represent the underwater acoustic sensor network as a directed graph, decompose the trusted routing into multiple single-hop trusted routing relay determination subproblems, and further transform it into a mathematical model that maximizes the average link quality under multiple constraints.

[0022] (1.1) UASN can be represented as a directed graph G=(V,E), where the vertex set V={ , ,..., } represents the set of sensor nodes in the network, and the set of edges E = {< i,j >| i,j∈ V represents the directed communication link between nodes. In the network routing process, three key node roles are defined: source node... That is, the sensor node currently receiving the data to be forwarded, and the relay node. (The next-hop node for data forwarding) and the destination node. In a multi-hop transmission mechanism, any node that has completed receiving data and is ready to forward it will be regarded as the new source node.

[0023] (1.2) UASN trusted routing aims to ensure the security of data transmission by ensuring the reliable selection of routing forwarding nodes. For example... Figure 2 Attacking nodes will engage in abnormal routing and data transmission, interfering with the normal operation of the UASN. Trusted routing effectively prevents transmitted data from being tampered with and prevents the leakage of sensitive routing information, thereby maintaining the integrity of routing data and path security. The UASN trusted routing problem is defined as determining the optimal route path for data packets from the source node to the destination node. This problem can be further decomposed into a single-hop trusted relay determination problem, that is, selecting trusted relay nodes hop-by-hop to ultimately construct an end-to-end secure routing path, which can be expressed as: for each source node... ∈V, select the optimal relay node from the set of candidate neighbor nodes. ∈N( A reliable forwarding link is constructed, and the process iterates until the data is forwarded to the target node, so as to achieve reliable data transmission for the entire network. (1.3) The trusted routing problem is transformed into the problem of maximizing average link quality, expressed as:

[0024] C1:

[0025] C2:

[0026] C3:

[0027] C4:

[0028] C5:

[0029] The overall goal is to maximize the average link quality within the UASN, aiming to determine secure and reliable routing paths for packet transmission within the UASN. Specifically, Indicates from the source node To relay node Link quality, This represents the total number of network nodes. Constraints C1 and C2 indicate that for each sensor node in a UASN, its data can only be relayed through a single neighboring node. Variables Indicator source node Does the data pass through the relay node? Forward. If yes The next hop relay node, then The value is 1; otherwise, the value is 0. Represents a node The set of neighbors. Constraint C3 defines the trust value range of all nodes as (0, 1). To ensure the security of data forwarding, constraint C4 requires relay nodes... The trust value must be greater than the trust threshold. Therefore, the final determined actual range of relay node trust values ​​is ( 1). Constraint C5 represents the calculation of link quality, where for arrive Error rate and packet loss rate of communication links.

[0030] Step (2): Calculate the trust value of the sensing node in the UASN by comprehensively considering the multi-dimensional trust features in the UASN. The trust value is used to represent the reliability of the node. (2.1) Trust values ​​are used to evaluate the credibility of nodes. The calculation of trust values ​​comprehensively considers four dimensions of trust characteristics: data credibility, communication behavior, energy state, and environmental factors. Considering that underwater anomalous nodes are usually highly concealed, few in number, and have behavioral characteristics that deviate significantly from normal nodes, the isolated forest algorithm is adopted as the core mechanism for trust assessment. This algorithm constructs an isolation tree based on a recursive random partitioning strategy. By repeatedly performing random partitioning on the trust feature data, all sample points are completely isolated, or the isolation tree reaches a preset maximum height. Under this mechanism, normal nodes in high-density areas require multiple partitions to be isolated, resulting in longer paths; while anomalous nodes in low-density areas, due to their characteristics deviating from the population distribution, are often quickly isolated, resulting in shorter paths. Therefore, the degree of anomalousness of a node can be quantified by its average path length in the isolated forest; that is, the shorter the path, the higher the probability of anomaly and the lower the trust value.

[0031] Step (3): As Figure 3 This diagram illustrates a trusted routing decision based on DRL. It enhances the reliability of routing relay by combining trust values ​​with an invalid action masking method. The trusted routing relay is determined based on the DRL algorithm, and the process is iterated until a complete routing path is determined to forward data to the target node. (3.1) Introducing DRL into the solution of the UASN trusted routing problem. Through continuous interaction with the underwater environment, trial and error and experience accumulation, DRL can gradually optimize the routing strategy and learn autonomously how to make optimal or near-optimal routing decisions in the highly dynamic and frequently changing topology of UASN, thereby effectively dealing with the inherent complexity, uncertainty and time-varying characteristics of UASN.

[0032] (3.2) The main elements of the DRL algorithm include the action space, state space, and reward function. (Time step) t Action space a t The state space contains all neighboring nodes of the current node. s t This includes the source node location, the destination node location, and the trust values ​​of all nodes in the UASN; the reward function focuses on two basic routing requirements: high security and short routing distance. Node definition. and The routing distance between the two nodes is the Euclidean distance, denoted as L. i, j Simultaneously, a distance variable ΔL is introduced to represent the difference in route path distance before and after selecting the relay node, specifically defined as ΔL = L r, d -L s, d By comparing ΔL, it can be clearly determined whether the relay node is closer to the destination node. The reward function is defined as:

[0033] A high positive feedback value C is given when the next hop directly reaches the destination node. The reward value is then calculated based on the reception status of the ACK confirmation message. If the source node receives the ACK message from the relay node within a predetermined time, it indicates that the data packet has been successfully received. At this point, the reward value is evaluated using three key metrics: link quality, normalized hop ratio, and distance ratio. Specifically, the first metric rewards the current link quality, while the second and third metric penalize excessive hop counts and distance from the destination node, respectively. The parameters η1, η2, and η3 achieve an adaptive trade-off between these metrics. Since it is generally expected that data will be successfully received by all nodes, negative feedback is given when an ACK message is not received.

[0034] (3.3) Based on the trust value obtained in (2), abnormal nodes can be filtered out when selecting the next action, thereby enhancing the reliability of routing relay nodes. Therefore, not all actions in the action space are valid. To address this issue, an invalid action masking method is used to limit the agent's selection range. During the training phase, the agent receives a mask that explicitly indicates valid and invalid actions, allowing it to select only from the set of valid actions. The invalid action masking method can completely prevent the agent from selecting invalid actions, thereby improving the stability and efficiency of the policy. Nodes in time slot τ v r The invalid action mask is defined as:

[0035] (3.4) Confirmation of trusted relay nodes based on the DRL algorithm. During training, the system observes the current state for each round. For each time step... t This generates an action mask. The invalid action mask method intervenes in the selection process by strategically modifying the Q-value, filtering out actions that are not allowed to be executed in the current state. , (3) Where ⊙ denotes element-wise multiplication. Using... A greedy strategy involves exploring or exploiting the resource with a certain probability. If the random number is less than the exploration rate... If the action is successful, the agent randomly selects an action from the set of valid actions; otherwise, the agent iterates through all valid actions and selects the action that maximizes the Q-value after masking, i.e., selects the most trusted relay node to forward the route. After executing the action, the agent observes the immediate reward r and transitions to the next state. This process continues to iterate until the data is forwarded to the target node.

[0036] In summary: This invention discloses a trusted routing protocol for underwater acoustic sensor networks (UASNs). The trusted routing protocol in UASNs is decomposed into multiple sub-problems and expressed as a mathematical problem of maximizing average link quality under multiple constraints. Then, the trust values ​​of sensor nodes in the UASN are calculated by comprehensively considering the multi-dimensional trust features, improving the reliability of candidate route relays. Combining the node trust values, an invalid action masking method is designed, and a deep reinforcement learning algorithm is used to determine the next-hop route relay node. This process is iterated until a trusted routing path from the source node to the target node is determined, completing the reliable transmission of underwater data. This protocol can improve the reliability of routing data transmission in UASNs and ensure UASN data security.

[0037] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A trusted routing protocol in an underwater acoustic sensor network, characterized in that, Includes the following steps: (1) The trusted routing problem of underwater acoustic sensor network (UASN) is decomposed into multiple single-hop trusted routing relay determination subproblems, and further transformed into an optimization model for optimizing average link quality under multiple constraints. (2) The trust value of the sensing node in the UASN is calculated by comprehensively considering the multidimensional trust features in the UASN. The trust value is used to represent the reliability of the node. (3) The Deep Reinforcement Learning (DRL) algorithm is used to implement the trusted relay decision and the invalid action masking strategy is used to eliminate potential unreliable nodes. The algorithm runs iteratively until a routing path from the source node to the target node is determined.

2. The trusted routing protocol in an underwater acoustic sensor network as described in claim 1, characterized in that, Step (1) specifically includes: (1.1) Represent UASN as a directed graph, where the set of vertices represents the set of sensor nodes in the network, the set of edges represents the directed communication links between nodes, and define three types of key node roles: source node is the sensor node that is currently forwarding data, relay node is the next hop node for data forwarding, and target node; (1.2) The UASN trusted routing problem is decomposed into an iterative single-hop trusted routing relay determination problem, that is, starting from the source node, the forwarding link is established by repeatedly selecting the best next-hop node from the candidate neighbors until the data is forwarded to the target node; (1.3) The objective function is to maximize the average link quality in the UASN, where the link quality is the ratio of the link quality from the source node to the relay node to the total number of network nodes.

3. The trusted routing protocol in an underwater acoustic sensor network as described in claim 1, characterized in that, Step (2) includes: The node trust evaluation comprehensively considers the four-dimensional characteristics of data, communication, energy and environment, and adopts the isolated forest algorithm as the core evaluation mechanism to address the sparsity and deviation of underwater abnormal nodes; an isolation tree is constructed by recursively random partitioning, and the average path length in the isolation forest is mapped inversely to the trust value by utilizing the statistical characteristics that abnormal nodes are easy to isolate and have short paths, thereby achieving accurate identification and trust quantification of hidden abnormal nodes.

4. The trusted routing protocol in an underwater acoustic sensor network as described in claim 1, characterized in that, Step (3) specifically includes: (3.1) Using the DRL algorithm, through its interaction and iteration with the underwater environment, it learns and optimizes its strategy autonomously under the condition of frequent topology changes, overcomes environmental uncertainty, and realizes the decision on the optimal trusted route; (3.2) The main elements of the DRL algorithm include the action space, state space, and reward function; the action space at time t a t The state space contains all neighboring nodes of the current node. s t This includes the source node location, the destination node location, and the trust values ​​of all nodes in the UASN. The reward function has three cases: if the node directly delivers the data to the destination, the system gives the highest positive reward; if it only successfully forwards the data to the relay node, while rewarding high link quality, it will penalize behaviors that generate too many hops or cause the location to move far away from the destination, thus balancing transmission efficiency; and if no acknowledgment message is received, the system will directly give a negative penalty, thereby guiding the route to avoid unreliable paths. (3.4) Introduce an invalid action mask to filter abnormal nodes; based on the trust value obtained in step (2), actions with a trust value greater than the threshold are considered valid, and those with a trust value less than the threshold are considered invalid. (3.5) Determine the trusted relay node for the next hop based on the DRL algorithm; this process is iterated until the complete routing path from the source node to the target node is determined.

Images