Underwater acoustic sensor network trust routing method based on environmental state perception

Abstract

The invention relates to an underwater acoustic sensor network trust routing method based on environmental state perception. The method comprises the following steps: step 1, constructing an environmental state perception model; and step 2, making a trust routing decision based on preferential sampling reinforcement learning. According to the underwater acoustic sensor network trust routing method based on environmental state perception provided by the invention, aiming at the dynamic nature of underwater network topology and the instability of underwater acoustic communication, the real-time environmental state perception and artificial intelligence driven routing decision are fused; and the reliability and robustness of data routing in dynamic and potential hostile underwater scenes are improved.

Description

Technical Field

[0001] The present invention relates to a trust routing method for an underwater acoustic sensor network based on environmental state perception, belonging to the technical field of underwater acoustic sensor network communication support. Background Art

[0002] Underwater acoustic sensor networks are a new paradigm formed to meet the integrated application requirements of complex underwater information perception, data transmission, and information processing. Among them, multi-hop forwarding-based underwater data routing is crucial for information acquisition. Underwater sensor networks can be widely applied to many fields such as marine disaster warning, hydrological information monitoring, marine resource exploration, underwater assisted navigation, and offshore military defense by deploying sensor nodes in the monitored water area and using underwater acoustic communication as the main medium for data transmission between underwater nodes. The open and unmanned characteristics of underwater acoustic sensor networks make it easy for normal nodes inside them to be invaded and compromised by malicious programs, and then turn into malicious nodes with legal network identities but performing network attack behaviors such as packet dropping, data content tampering, and high-frequency interactive requests. The trust mechanism can establish a prediction model based on the historical interaction experience of both parties in the interaction, so as to use the obtained trust evidence to predict and evaluate the future behavior of the evaluation object and avoid the threat of malicious attacks in advance. Therefore, constructing a trust routing for underwater acoustic sensor networks is an important means to ensure underwater data security. However, the dynamic characteristics of the underwater network topology and the instability of underwater acoustic communication pose significant challenges to achieving efficient and reliable data transmission.

[0003] In order to design a high-quality routing scheme for underwater acoustic sensor networks, researchers at home and abroad have proposed various solutions, and the relevant literature is as follows:

[0004] 1. In 2021, Zhang et al. proposed an opportunistic routing protocol (RLOR) that combines the advantages of opportunistic routing and reinforcement learning in "Reinforcement learning-based opportunistic routing protocol for underwater acoustic sensor networks". This protocol comprehensively evaluates the environment around the node and selects a suitable relay node, which is a distributed routing method. It innovatively introduces a recovery mechanism, enabling packets to effectively bypass the void area and continue to be forwarded, significantly improving the data delivery rate in sparse networks.

[0005] 2. In 2023, Shen et al. proposed an adaptive green power control-aided routing protocol (PCAQR) in "A power control-aided q-learning-based routing protocol for optical-acoustic hybrid underwater sensor networks", which consists of two core stages: power control and data packet forwarding. The former realizes the adaptive adjustment of the node transmission power based on the potential game theory, and the latter dynamically optimizes the forwarding strategy through the Q-learning technology. By jointly optimizing the energy efficiency, the success rate of data packet forwarding and the network connectivity, this protocol solves the dual challenges of limited energy and unstable links in underwater networks.

[0006] 3. In 2024, Zhu et al. designed an adaptive routing algorithm (GHL-SAR) for the characteristics of underwater acoustic communication in "An efficient secure and adaptive routing protocol based on GMM-HMM-LSTM for internet of underwater things". This algorithm uses the Gaussian mixture model-hidden Markov model (GMM-HMM) to predict the potential hidden state sequence, and reveals the relationship between the state and the trust level through the long short-term memory network (LSTM). At the same time, a link quality evaluation mechanism is constructed based on the particle swarm optimization algorithm, and finally a dynamic routing decision under multiple constraints is realized.

[0007] 4. In 2024, Wang et al. proposed a trust routing algorithm (GTR) based on the generative adversarial network (GAN) in "Gtr: Gan-based trusted routing algorithm for underwater wireless sensor networks" to address the problem that underwater data is vulnerable to malicious attacks. This algorithm constructs an underwater node trust evaluation model by defining trust feature attributes and evaluation matrices, and uses the generative adversarial training characteristics of GAN to enhance the ability to detect malicious nodes for unlabeled and imbalanced data. Experiments show that even under high-intensity attacks, GTR can still maintain a detection accuracy of over 90%. Summary of the Invention

[0008] The technical problem to be solved by the present invention is as follows: Aiming at the dynamics of underwater network topology and the instability of underwater acoustic communication, which pose significant challenges to efficient and reliable underwater data transmission, a trust routing method for underwater acoustic sensor networks based on environmental state perception is proposed. By integrating real-time environmental state perception and artificial intelligence-driven routing decisions, the reliability and robustness of data routing in dynamic and potentially hostile underwater scenarios are enhanced. First, a dynamic topology perception model and an underwater acoustic communication perception model are established to achieve real-time perception of the interaction context, and then continuous input is provided for the deep reinforcement learning agent, enabling it to dynamically adapt to topology changes and communication conditions, thereby promoting priority experience sampling during the training process of the routing decision model and indirectly improving the model training efficiency and routing decision accuracy.

[0009] To achieve the above object, the present invention is implemented through the following technical solutions:

[0010] A trust routing method for underwater acoustic sensor networks based on environmental state perception, comprising the following steps:

[0011] Step 1: Construction of the environmental state perception model

[0012] The environmental state perception model consists of two parts: a node topology perception module and an underwater acoustic communication perception module;

[0013] The node topology perception module uses the meandering ocean current modeling idea. By inputting the current position information of the node, the predicted value of the node position after a specified time slot can be obtained; then, the topology dynamic factor F1 is calculated based on the deviation between the current position and the predicted position of the node;

[0014] The underwater acoustic communication perception module first obtains the signal-to-noise ratio in the current communication environment of the node through the passive sonar equation, and then calculates the relative difference between the signal-to-noise ratio and the detection threshold to obtain the underwater acoustic communication factor F2;

[0015] Step 2: Trust routing decision based on priority sampling reinforcement learning

[0016] The trust routing based on priority sampling reinforcement learning is divided into two modules: sample pool priority ranking and reinforcement learning trust routing;

[0017] First, the sampled sample data is saved in the experience pool, and the samples in the experience pool are sorted according to the rule that the lower the values of F1 and F2, the higher the priority;

[0018] The reinforcement learning trust routing policy collects samples from the experience pool in batches according to the priority, and uses the deep reinforcement learning algorithm for training until the training converges; then, the converged policy network is used to guide the node to forward data packets, and finally a trustworthy underwater routing is constructed.

[0019] Preferably, the specific steps of the above step 1 are as follows:

[0020] 1-1) The node topology perception module first uses an improved meandering flow model to describe the node movement law:

[0021]

[0022] where B(t) = A + ∈cos(ωt) controls the flow field width, k characterizes the flow field tortuosity, and c is the phase velocity;

[0023] Then, the node movement velocity field is obtained by taking the partial derivatives of the stream function ψ with respect to x and y respectively:

[0024]

[0025] Next, by inputting the current position information of the node, a node position prediction model is established:

[0026]

[0027] Finally, according to the deviation between the current position and the predicted position of the node, the topological dynamic factor F1 is obtained:

[0028]

[0029] where represents the initial distance deviation between two nodes, represents the predicted distance deviation between two nodes;

[0030] 1-2) The underwater acoustic communication perception module first uses the passive sonar equation to evaluate the channel state:

[0031] SNR = SL - TL - NL + DI ≥ DT

[0032] where: SNR is the signal-to-noise ratio, SL is the sound source level, TL = k×10logr + αr×10 -3 Calculate the propagation loss, DI and DT are the underwater acoustic signal direction factor and detection threshold respectively, calculate the comprehensive noise, where N i is the power spectral density of a certain type of noise, and τ represents the number of noise types;

[0033] Then, by calculating the relative difference between the signal-to-noise ratio and the detection threshold, the underwater acoustic communication factor F2 is obtained:

[0034]

[0035] Preferably, the trust routing decision method based on priority sampling reinforcement learning in the above step 2 is as follows:

[0036] 2-1) The sample priority sorting module first adds the outputs F1 and F2 of the environmental perception model to each sample, so that the experience e at time t t is represented by the following tuple:

[0037] e t =(s t , a t , r t , s t+1 , F 1,t , F 2,t )

[0038] where s t represents the state, a t represents the action, r t represents the reward, s t+1 represents the state at the next moment;

[0039] Then, the priority function φ(·) is used to calculate the priority of each experience:

[0040] φ(e t )=-(w1F 1,t + w2F 2,t )

[0041] where w1 and w2 are the weights of two types of environmental perception factors respectively;

[0042] Next, the experience pool D is sorted in descending order according to the calculated priority, and the sorted experience pool D sorted :

[0043] D sorted = sort(D, φ(e t ), descending)

[0044] 2-2) The reinforcement learning routing decision module first selects the first 2*N experiences from the sorted experience pool to form a new experience pool D new :

[0045] D new ={e ∈ D sorted | e is in the first 2*N positions}

[0046] Then, N experiences are randomly sampled from the new experience pool D new to form a batch of samples B:

[0047] B = {e1, e2,..., e N} ~ Uniform(D new )

[0048] Next, the batch of samples B is used to train the policy network μ(s; θ):

[0049]

[0050] where L μ represents the loss function, Q and μ represent the value network and the policy network respectively, and θ Q , θ μ correspond to the parameters of these two networks respectively, and λ μ is the adjustment coefficient, refers to performing gradient descent on the function L μ with respect to θ μ .

[0051] Repeat the above training process until the policy network and the value network converge. Output the converged policy network μ(s; θ μ ) as the routing policy, and obtain the forwarding target node according to this policy, thereby constructing a trust routing for the underwater acoustic sensor network. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a schematic diagram of the environmental state perception model of the present invention;

[0053] Figure 2 is a schematic diagram of the trust routing decision method based on priority sampling reinforcement learning of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0054] The present invention will be further described in detail below with reference to the drawings and embodiments.

[0055] A trust routing method for an underwater acoustic sensor network based on environmental state perception, the steps of which include:

[0056] Step 1: Construction of the environmental state perception model

[0057] As Figure 1 shown, the environmental state perception model consists of a node topology perception module and an underwater acoustic communication perception module.

[0058] 1) The node topology perception module first uses an improved meandering flow model to describe the node movement law:

[0059]

[0060] where B(t) = A + ∈cos(ωt) controls the flow field width, k characterizes the flow field tortuosity, and c is the phase velocity.

[0061] Then, the node movement velocity field is obtained by taking the partial derivatives of the stream function ψ with respect to x and y respectively:

[0062]

[0063] Next, a node position prediction model is established based on the current position information of the input node:

[0064]

[0065] Finally, according to the deviation between the current position and the predicted position of the node, the topological dynamic factor F1 is obtained:

[0066]

[0067] where represents the initial distance deviation between two nodes, represents the predicted distance deviation between two nodes.

[0068] 2) The underwater acoustic communication perception module first uses the passive sonar equation to evaluate the channel state:

[0069] SNR = SL - TL - NL + DI ≥ DT

[0070] where: SL is the sound source level, TL = k × 10logr + αr × 10 -3 calculates the propagation loss, NL = 10log(N t + N s + N w + N th ) calculates the comprehensive noise.

[0071] Then, by calculating the relative difference between the signal-to-noise ratio and the detection threshold, the underwater acoustic communication factor F2 is obtained:

[0072]

[0073] Step 2: Trust routing decision based on prioritized sampling reinforcement learning

[0074] As Figure 2 shown, the trust routing decision based on prioritized sampling reinforcement learning is divided into two parts: the sample priority ranking module and the reinforcement learning routing decision module.

[0075] 1) The sample priority ranking module first adds the output of the environmental perception model to each sample, so that each experience e t is represented by the following tuple:

[0076] e t = (s t , a t , r t , s t+1 , F 1,t , F 2,t )

[0077] where s t represents the state, a t represents the action, rt Indicates a reward, s t+1 Indicates the state at the next moment;

[0078] Then, use the priority function φ(·) to calculate the priority of each experience:

[0079] φ(e t ) = -(w1F 1,t + w2F 2,t )

[0080] where w1 and w2 are the weights of two types of environmental perception factors respectively;

[0081] Next, sort the experience pool D in descending order according to the calculated priority to obtain the sorted experience pool D sorted :

[0082] D sorted = sort(D, φ(e t ), descending)

[0083] 2) The reinforcement learning routing decision module first selects the first 2*N experiences from the sorted experience pool to form a new experience pool D new :

[0084] D new = {e ∈ D sorted | e is in the first 2*N positions}

[0085] Then randomly sample N experiences from the new experience pool D new to form a batch sample B:

[0086] B = {e1, e2,..., e N} ~ Uniform(D new )

[0087] Next, use the batch sample B to train the policy network μ(s; θ):

[0088]

[0089] Repeat the above training process until the policy network and the value network converge. Output the converged policy network μ(s; θ μ ) as the routing policy, and obtain the forwarding target node according to this policy, so as to construct a trust routing for the underwater acoustic sensor network.

Claims

1. An underwater acoustic sensor network trust routing method based on environmental state perception, characterized in that: It includes the following steps: Step 1: Construction of the environmental state perception model The environmental state perception model consists of two parts: the node topology perception module and the underwater acoustic communication perception module; The node topology perception module uses the meandering ocean current modeling idea. By inputting the current position information of the node, the predicted value of the node position after a specified time slot can be obtained; then, the topology dynamic factor F1 is calculated according to the deviation between the current position and the predicted position of the node; The underwater acoustic communication perception module first obtains the signal-to-noise ratio in the current communication environment of the node through the passive sonar equation, and then obtains the underwater acoustic communication factor F2 by calculating the relative difference between the signal-to-noise ratio and the detection threshold; Step 2: Trust routing decision based on prioritized sampling reinforcement learning The trust routing based on prioritized sampling reinforcement learning is divided into two modules: sample pool priority ranking and reinforcement learning trust routing; First, the sampled sample data is saved in the experience pool, and the samples in the experience pool are sorted according to the rule that the lower the values of F1 and F2, the higher the priority.

2. The enhanced learning trust routing policy collects samples batch by batch from the experience pool according to the priority, and uses the deep reinforcement learning algorithm for training until the training converges; then, the converged policy network is used to guide the nodes to forward data packets, and finally a trustworthy underwater routing is constructed. The method for trust routing in an underwater acoustic sensor network based on environmental state perception according to claim 1, wherein: The specific steps of Step 1 are as follows: 1-1) The node topology perception module first uses an improved meandering flow model to describe the node motion law: where B(t) = A + ∈cos(ωt) controls the flow field width, k characterizes the flow field tortuosity, and c is the phase velocity; Then, the node motion velocity field is obtained by taking the partial derivatives of the stream function ψ with respect to x and y respectively: Next, by inputting the current position information of the node, a node position prediction model is established: Finally, according to the deviation between the current position and the predicted position of the node, the topology dynamic factor F1 is obtained: Among them represents the initial distance deviation between two nodes represents the predicted distance deviation between two nodes 1-2) The underwater acoustic communication perception module first uses the passive sonar equation to evaluate the channel state: SNR = SL - TL - NL + DI ≥ DT Where: SNR is the signal-to-noise ratio, SL is the sound source level, TL = k × 10logr + αr × 10 -3 Calculate the propagation loss. DI and DDT are the underwater acoustic signal direction factor and the detection threshold respectively, Calculate the comprehensive noise, where N i is the power spectral density of a certain type of noise, and τ represents the number of noise types; Then, the underwater acoustic communication factor F2 is obtained by calculating the relative difference between the signal-to-noise ratio and the detection threshold:

3. The trust routing method for an underwater acoustic sensor network based on environmental state perception according to claim 2, wherein: In Step 2, the trust routing decision method based on prioritized sampling reinforcement learning is as follows: 2-1) The sample priority sorting module first adds the outputs F1 and F2 of the environmental perception model to each sample, so that the experience e at time t t is represented by the following tuple: e t = (s t , a t , r t , s t+1 , F 1,t , F 2,t ) where s t represents the state, a t represents the action, r t represents the reward, s t+1 represents the state at the next moment; Then, the priority function φ(·) is used to calculate the priority of each experience: φ(e t ) = -(w1F 1,t + w2F 2,t ) where w1 and w2 are the weights of the two types of environmental perception factors respectively; Next, the experience pool D is sorted in descending order according to the calculated priority to obtain the sorted experience pool D sorted : D sorted = sort(D, φ(e t ), descending) 2-2) The reinforcement learning routing decision-making module first selects the first 2*N experiences from the sorted experience pool to form a new experience pool D new : D new = {e ∈ D sorted | e is in the first 2 * N positions} Then randomly sample N experiences from the new experience pool D new to form a batch sample B: B = {e1, e2, …, e N} ~ Uniform(D new ) Next, the policy network μ(s; θ) is trained using the batch sample B: Among which L μ represents the loss function, Q and μ represent the value network and the policy network respectively, and θ Q , θ μ correspond to the parameters of these two networks respectively, and λ μ is the adjustment coefficient, which means to perform gradient descent on the function L μ with respect to θ μ ; Repeat the above training process until the policy network and the value network converge; output the converged policy network μ(s; θ μ ) as the routing policy, and obtain the forwarding target node according to this policy, so as to construct a trust routing for the underwater acoustic sensor network.

Images