A method, device and medium for determining routing of an underwater acoustic sensor network

Abstract

The application discloses a routing determination method and device of an underwater acoustic sensor network and a medium, relates to the field of underwater acoustic communication and network routing optimization, and comprises the following steps: based on the underwater acoustic sensor network, taking the negative value of an offline link cost matrix as a reward function, and adopting a Q-learning algorithm to perform offline iterative training on an initial Q table, an optimized Q table is obtained; based on an online initial link cost matrix and the optimized Q table, a remodeling formula is adopted to calculate a remodeled online initial link cost matrix; based on an online initial adjacency matrix and the remodeled online initial link cost matrix, a Dijkstra algorithm is adopted to determine an optimal path of the underwater acoustic sensor network, and the determination of the routing of the underwater acoustic sensor network is completed. The application optimizes the balance among the delivery rate, the time delay and the throughput, and improves the adaptability and robustness of data transmission of the underwater acoustic sensor network.

Description

Technical Field

[0001] This application relates to the field of underwater acoustic communication and network routing optimization technology, and in particular to a routing determination method, device and medium for underwater acoustic sensor networks. Background Technology

[0002] With the continuous development of marine monitoring and seabed exploration technologies, underwater acoustic sensor networks have become a key infrastructure for acquiring deep-sea data. However, underwater acoustic channels have inherent challenges such as limited bandwidth, large propagation delay, severe signal attenuation, and time-varying link states, making it extremely difficult to achieve reliable, low-latency, and high-efficiency data routing in this environment.

[0003] In some situations, while classic shortest path routing algorithms (such as Dijkstra's algorithm) have advantages such as low computational complexity, strong path determinism, and ease of deployment, their decisions often rely on instantaneous or pre-set fixed link costs. When facing dynamically changing underwater environments, due to the lack of "historical prior" guidance, they are prone to blindly choosing fragile links with seemingly low instantaneous costs but unstable connections. This not only increases end-to-end transmission latency but also consumes limited underwater acoustic bandwidth, ultimately leading to a significant decrease in the overall packet delivery rate and throughput of the network. Although online learning routing methods can improve performance, online learning algorithms typically face extremely high convergence delays and require substantial online protocol overhead. This undermines the plug-and-play nature of classic shortest path routing algorithms, making it difficult to achieve optimal overall data delivery rate, latency, and throughput in underwater acoustic sensor networks. Summary of the Invention

[0004] The purpose of this application is to provide a routing determination method, device, and medium for underwater acoustic sensor networks, which improves the adaptability and robustness of data transmission in underwater acoustic sensor networks by optimizing the balance between delivery rate, latency, and throughput.

[0005] To achieve the above objectives, this application provides the following solution.

[0006] In a first aspect, this application provides a routing determination method for an underwater acoustic sensor network, including: Based on an underwater acoustic sensor network, the negative value of the offline link cost matrix is ​​used as the reward function, and the Q-learning algorithm is used to perform offline iterative training on the initial Q-table to obtain an optimized Q-table; the offline link cost matrix is ​​constructed based on offline latency, offline energy consumption, and offline packet loss rate. The online initial link cost matrix and online initial adjacency matrix of the underwater acoustic sensor network are obtained; the online initial link cost matrix is ​​constructed based on online latency, online energy consumption, and online packet loss rate. Based on the online initial link cost matrix and the optimized Q table, the reshaped online initial link cost matrix is ​​calculated using the reshaping formula. Based on the online initial adjacency matrix and the reshaped online initial link cost matrix, the Dijkstra algorithm is used to determine the optimal path of the underwater acoustic sensor network, thus completing the routing determination of the underwater acoustic sensor network.

[0007] Secondly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described routing determination method for an underwater acoustic sensor network.

[0008] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described routing determination method for an underwater acoustic sensor network.

[0009] According to the specific embodiments provided in this application, this application has the following technical effects: This application introduces an offline link cost matrix (constructed based on offline latency, offline energy consumption, and offline packet loss rate) and uses the Q-learning algorithm to iteratively train the initial Q-table offline to obtain an optimized Q-table. The optimized Q-table is then used to reshape the online initial link cost matrix, integrating historical routing experience with real-time link status, thus optimizing delivery rate, latency, and throughput. Based on the reshaped online initial link cost matrix and the online initial adjacency matrix, the Dijkstra algorithm is used to obtain the optimal path for the underwater acoustic sensor network. This approach not only retains the determinism, polynomial time complexity, and plug-and-play characteristics of the Dijkstra algorithm but also enables the Q-table to optimize the delivery rate, latency, and throughput of the underwater acoustic sensor network, improving the adaptability and robustness of data transmission in the underwater acoustic sensor network. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A flowchart illustrating the routing determination method for the underwater acoustic sensor network provided in this application.

[0012] Figure 2A schematic diagram illustrating the process of obtaining the Q-table through offline iterative training of the Q-learning algorithm provided in this application.

[0013] Figure 3 This is a flowchart illustrating the routing determination method for underwater acoustic sensor networks in practical applications provided in this application.

[0014] Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0016] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0017] In one exemplary embodiment, such as Figure 1 As shown, a routing determination method for an underwater acoustic sensor network is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is described using a server as an example, and includes the following steps S1 to S8.

[0018] Step S1: Based on the underwater acoustic sensor network, using the negative value of the offline link cost matrix as the reward function, the Q-learning algorithm is used to perform offline iterative training on the initial Q-table to obtain the optimized Q-table. The offline link cost matrix is ​​constructed based on offline latency, offline energy consumption, and offline packet loss rate. The numerical values ​​of each element in the optimized Q-table represent the long-term communication quality between every two nodes in the underwater acoustic sensor network.

[0019] Specifically, the Q-learning algorithm is used in the offline phase. The negative value of the offline link cost matrix is ​​used as the reward function, and the optimized Q table is trained by temporal difference update method. As a historical experience prior, it reflects the long-term utility of choosing this link as the next hop.

[0020] As an feasible approach, such as Figure 2 As shown, step S1 specifically includes steps S11 to S15: Step S11: Obtain the initial Q table.

[0021] Step S12: Obtain all offline underwater nodes of the underwater acoustic sensor network and the offline network performance indicators between each offline underwater node; the offline network performance indicators include offline latency, offline power consumption and offline packet loss rate.

[0022] Step S13: Randomly select all offline underwater nodes to determine the offline source node and offline destination node.

[0023] Step S14: Based on the initial Q-table, offline source node, offline destination node, and offline network performance indicators among each offline underwater node, adopt... A greedy strategy is used to perform offline iterative training of the Q-learning algorithm to obtain an updated initial Q-table.

[0024] As one possible implementation, step S14 specifically includes steps S141 to S144: Step S141: Determine the relay node for the current hop; the first relay node is the offline source node.

[0025] Step S142: Based on the relay node of the current hop and the initial Q table, use... A greedy strategy is used to select the neighbor node to hop to.

[0026] Step S143: Based on the offline network performance metrics between the relay node and neighboring nodes of the current hop, construct the offline link cost matrix between the relay node and neighboring nodes of the current hop.

[0027] As an feasible approach, the expression for the offline link cost matrix is: .

[0028] in, The offline link cost matrix is ​​the first... The node and the first Offline link cost between nodes; , and All are weighting coefficients; For the first The node and the first Offline latency between nodes; For the first The node and the first Offline energy consumption between nodes; For the first The node and the first Offline packet loss rate between nodes.

[0029] Specifically, the offline link cost matrix Let be a symmetric nonnegative matrix, satisfying =0. In the actual simulation and modeling of underwater acoustic sensor networks, the offline link cost matrix is... It is not a simple distance, but a multi-objective weighted function based on the characteristics of the underwater channel (usually including offline latency, offline energy consumption and offline packet loss rate).

[0030] ,in, For the first The node and the first Euclidean distance between nodes The speed of sound underwater, such as 1500 m / s.

[0031] It is the transmission energy consumption required by the transmitter to overcome path loss and absorption attenuation, based on the Thorp attenuation model.

[0032] Indicates if the first The node and the first If the distance between nodes exceeds the maximum communication radius, or if the link is congested, a very large penalty value is imposed (e.g., (or a very large positive number), indicating that the link is not feasible.

[0033] Step S144: Based on the offline link cost matrix, update the initial Q table using time-series difference until the current hop's neighbor node is the offline destination node, and obtain the updated Q table.

[0034] As an feasible approach, the updated expression for the Q table is: .

[0035] .

[0036] in, For the updated Q table, the first The node and the first The Q-value between nodes; For the initial Q table, the first The node and the first The Q-value between nodes; The learning rate; For the first The node and the first The reward function between nodes; Discount factor; For the first The set of all feasible neighbor nodes of a node within the effective underwater acoustic communication range; For the initial Q table, the first The node and the first The Q-value between nodes; The offline link cost matrix is ​​the first... The node and the first Offline link cost between nodes.

[0037] Step S15: When the updated initial Q-table converges, the optimized Q-table is obtained.

[0038] Specifically, the Q-learning algorithm models next-hop routing as a round-robin Markov decision process, where the state is the relay node of the current hop and the action is the neighbor node of the current hop; the reward function is defined as... ;use A greedy strategy balances exploration and exploitation, i.e., using probability. Randomly select an action from feasible neighbor nodes to explore, with a probability of 1- The action with the largest current Q value is selected for utilization, and the Q value is iteratively updated using a time-difference update rule. The Q-table is initialized using a heuristic inversely proportional to the link cost matrix. The values ​​are then cropped to a bounded range to ensure numerical stability. After training, the resulting optimized Q-table will be stored as offline prior knowledge and used for reshaping the subsequent online initial link cost matrix.

[0039] Step S2: Obtain the online initial link cost matrix and online initial adjacency matrix of the underwater acoustic sensor network; the online initial link cost matrix is ​​constructed based on online latency, online energy consumption and online packet loss rate.

[0040] As one feasible approach, step S2 specifically includes steps S21 to S23: Step S21: Obtain all online underwater nodes of the underwater acoustic sensor network and the online network performance indicators between each online underwater node; the online network performance indicators include online latency, online power consumption and online packet loss rate; the online underwater nodes include: online source nodes, online relay nodes and online destination nodes.

[0041] Step S22: Construct a bidirectional weighted graph based on all online underwater nodes and online network performance metrics.

[0042] Step S23: Based on the bidirectional weighted graph, determine the online initial link cost matrix and the online initial adjacency matrix.

[0043] Specifically, the network is modeled as a bidirectional weighted graph, and the link cost matrix and adjacency matrix are initialized. The bidirectional weighted graph is represented as follows: ,in For a set of nodes, Let C be the set of edges; the link cost matrix C is a symmetric nonnegative matrix that satisfies the following condition: The adjacency matrix A is a symmetric binary matrix. This indicates that there is a feasible link between node i and node j.

[0044] Step S3: Based on the online initial link cost matrix and the optimized Q table, calculate the reshaped online initial link cost matrix using the reshaping formula.

[0045] Specifically, during online routing, real-time link information is received through a lightweight wrapper, the Q-values ​​of effective edges are normalized, and the original link costs are reshaped in a bounded multiplicative manner with an adjustable scaling factor to obtain an empirically enhanced cost matrix.

[0046] As one feasible approach, step S3 specifically includes steps S31 to S32: Step S31: Normalize the optimized Q-table to obtain the normalized optimized Q-table.

[0047] Specifically, the optimized Q-table is obtained by performing min-max normalization on the Q-value. ,in Then through the formula Calculate the reshaped online initial link cost matrix.

[0048] Step S32: Based on the online initial link cost matrix and the normalized optimized Q table, the reshaped formula is used to calculate the reshaped online initial link cost matrix.

[0049] Specifically, the reshaping formula applies bounded multiplicative correction to the online initial link cost matrix, thereby generating a cost matrix that integrates historical experience and real-time status (i.e., the reshaped online initial link cost matrix). This formula achieves intelligent enhancement of routing decisions without modifying the underlying communication protocol.

[0050] As an feasible approach, the reshaping formula is expressed as follows: .

[0051] in, The first in the reshaped online initial link cost matrix The node and the first The cost of online links between nodes; The first in the online initial link cost matrix The node and the first The cost of online links between nodes; This is a truncation function used to limit the calculated scaling factor to a specified upper and lower limit range; The prior strength coefficient; For the optimized Q-table after normalization, the first... The node and the first The Q-value between nodes; This is the lower limit of the scaling factor, usually set to a constant greater than 0, to ensure that the cost of the reshaped link is non-negative and not too small; This is the upper limit of the scaling factor, used to prevent excessive reliance on historical Q tables from causing costs to be abnormally amplified. The function limits the scaling factor to Within the range to ensure that all weights are non-negative and to avoid excessive bias.

[0052] Step S4: Based on the online initial adjacency matrix and the reshaped online initial link cost matrix, the Dijkstra algorithm is used to determine the optimal path of the underwater acoustic sensor network, thus completing the routing determination of the underwater acoustic sensor network.

[0053] Specifically, the online initial adjacency matrix and the reshaped online initial link cost matrix are taken as input, and the standard Dijkstra shortest path algorithm solver without modification is directly called. The solver performs a deterministic shortest path search based on the input reshaped online initial link cost matrix and outputs the optimal route from the source node to the destination node with the minimum cumulative reshaping cost, thus obtaining the optimal path of the underwater acoustic sensor network.

[0054] In practical applications, there may be situations where the Q-table fails to load or cannot be loaded at all. When this occurs, Dijkstra's algorithm is used based on the online initial link cost matrix and the online initial adjacency matrix to obtain the optimal path for the underwater acoustic sensor network, thus determining the route and ensuring the robustness of the routing function. Therefore, a judgment logic needs to be added to handle this situation, such as... Figure 3 As shown, starting from network modeling and initialization, the system first enters the offline Q-learning training phase. It uses the Q-learning algorithm for offline iterative training to generate and store an optimized Q-table based on edge-level empirical priors. When an online routing request is triggered, real-time link information is loaded (determining the source and destination nodes), and the availability of the optimized Q-table is checked. If available, the optimized Q-table is normalized, and the link costs are reshaped. The reshaped cost matrix (i.e., the reshaped initial online link cost matrix) and adjacency matrix are then input into the standard Dijkstra solver to calculate the path. If unavailable, the system automatically reverts to the classic Dijkstra algorithm. Finally, the system outputs the optimal path and executes data forwarding, completing the routing decision.

[0055] To further verify the practical effectiveness of the proposed method, a comparative test was conducted with the classic Dijkstra routing algorithm in a typical three-dimensional sparse underwater acoustic sensor network scenario (including seabed source nodes, underwater relay nodes, and surface convergence nodes, with long propagation delays and time-varying link attenuation). Test data shows that by introducing an optimized Q-table based on offline Q-value priors and an online initial link cost matrix reshaping mechanism, the proposed method effectively avoids "trap" links that have low local instantaneous costs but poor long-term reliability. Under the same network environment configuration, compared to the classic Dijkstra algorithm: 1. In terms of delivery reliability, experience-based guidance was used to avoid blind forwarding, resulting in a relative improvement of approximately 13.7% in packet delivery rate (PDR).

[0056] 2. In terms of transmission efficiency, due to the reduction in retransmissions and the improvement in link utilization, network throughput achieved a significant increase of approximately 23.4%.

[0057] 3. Regarding communication latency, thanks to more stable path selection, the average end-to-end delivery latency of data packets has been reduced by approximately 11.2%.

[0058] The experimental results fully verify that the present invention can significantly improve the overall communication performance of underwater acoustic networks with extremely low online computational overhead without changing the existing protocol stack.

[0059] Further simulation comparison experiments demonstrate that, in complex and dynamic underwater acoustic environments, this application not only retains the deterministic and polynomial time complexity advantages of Dijkstra's algorithm but also achieves significant optimizations in various key communication metrics. Specifically, compared to the classic Dijkstra shortest path algorithm that directly uses the original link cost matrix, this application improves the packet delivery rate (PDR) of underwater sensor networks by approximately 13.7%, increases the total network throughput by approximately 23.4%, and reduces the average end-to-end latency of data packets by approximately 11.2%. These quantitative indicators fully demonstrate that this application achieves a better trade-off between delivery rate, latency, and throughput in underwater acoustic communication.

[0060] The advantages of the routing determination method for underwater acoustic sensor networks proposed in this application are mainly reflected in the following aspects: 1. By introducing an offline link cost matrix (constructed based on offline latency, offline energy consumption, and offline packet loss rate), the Q-learning algorithm is used to perform offline iterative training on the initial Q-table, realizing edge-level Q-prior learning. Then, the online initial link cost matrix is ​​adjusted accordingly, and then solved using the Dijkstra algorithm. This retains the advantages of Dijkstra's determinism and polynomial time complexity, while realizing experience-guided path selection.

[0061] 2. Compared with traditional Dijkstra routing, it has significant improvements in packet delivery rate, throughput and end-to-end latency, achieving a better "latency-reliability-throughput" trade-off. It significantly improves the adaptability and robustness of underwater acoustic sensor network data transmission, as well as the communication reliability and energy efficiency of underwater acoustic sensor networks in complex marine environments. It is widely used in marine environmental monitoring, seabed resource exploration and marine ecological research.

[0062] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 4 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores Q-tables. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a routing determination method for an underwater acoustic sensor network.

[0063] Those skilled in the art will understand that Figure 4 The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0064] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0065] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0066] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0067] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, etc., and are not limited to these.

[0068] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0069] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A routing determination method for an underwater acoustic sensor network, characterized in that, include: Based on an underwater acoustic sensor network, the negative value of the offline link cost matrix is ​​used as the reward function, and the Q-learning algorithm is used to perform offline iterative training on the initial Q-table to obtain an optimized Q-table; the offline link cost matrix is ​​constructed based on offline latency, offline energy consumption, and offline packet loss rate. The online initial link cost matrix and online initial adjacency matrix of the underwater acoustic sensor network are obtained; the online initial link cost matrix is ​​constructed based on online latency, online energy consumption, and online packet loss rate. Based on the online initial link cost matrix and the optimized Q table, the reshaped online initial link cost matrix is ​​calculated using the reshaping formula. Based on the online initial adjacency matrix and the reshaped online initial link cost matrix, the Dijkstra algorithm is used to determine the optimal path of the underwater acoustic sensor network, thus completing the routing determination of the underwater acoustic sensor network. Specifically, based on an underwater acoustic sensor network, the negative value of the offline link cost matrix is ​​used as the reward function, and the Q-learning algorithm is employed to iteratively train the initial Q-table offline to obtain an optimized Q-table, which includes: Obtain the initial Q table; Acquire all offline underwater nodes of the underwater acoustic sensor network and the offline network performance indicators between each offline underwater node; the offline network performance indicators include offline latency, offline power consumption, and offline packet loss rate. Randomly select all offline underwater nodes to determine the offline source node and offline destination node; Based on the initial Q-table, the offline source node, the offline destination node, and the offline network performance metrics among the offline underwater nodes, the following is adopted: A greedy strategy is used to perform offline iterative training of the Q-learning algorithm to obtain an updated initial Q-table. When the updated initial Q-table converges, the optimized Q-table is obtained; The expression for the offline link cost matrix is: ； in, The offline link cost matrix is ​​the first... The node and the first Offline link cost between nodes; , and All are weighting coefficients; For the first The node and the first Offline latency between nodes; For the first The node and the first Offline energy consumption between nodes; For the first The node and the first Offline packet loss rate between nodes.

2. The routing determination method for underwater acoustic sensor networks according to claim 1, characterized in that, Based on the initial Q-table, the offline source node, the offline destination node, and the offline network performance metrics among the offline underwater nodes, the following is adopted: A greedy strategy is used to perform offline iterative training of the Q-learning algorithm to obtain an updated initial Q-table, which includes: Determine the relay node for the current hop; the first relay node is the offline source node; Based on the relay node of the current hop and the initial Q table, using A greedy strategy is used to select and determine the neighboring node for the current hop; Based on the offline network performance metrics between the relay node and neighbor nodes of the current hop, construct the offline link cost matrix between the relay node and neighbor nodes of the current hop. Based on the offline link cost matrix, the initial Q-table is updated using time-series difference until the current hop's neighbor node is the offline destination node, thus obtaining the updated Q-table.

3. The routing determination method for underwater acoustic sensor networks according to claim 2, characterized in that, The updated expression for the Q table is: ； ； in, For the updated Q table, the first The node and the first Q-values ​​between nodes; For the initial Q table, the first The node and the first Q-values ​​between nodes; The learning rate; For the first The node and the first The reward function between nodes; Discount factor; For the first The set of all feasible neighbor nodes of a node within the effective underwater acoustic communication range; For the initial Q table, the first The node and the first Q-values ​​between nodes; The offline link cost matrix is ​​the first... The node and the first Offline link cost between nodes.

4. The routing determination method for underwater acoustic sensor networks according to claim 1, characterized in that, Based on the online initial link cost matrix and the optimized Q-table, the reshaped online initial link cost matrix is ​​calculated using the reshaping formula, specifically including: The optimized Q-table is normalized to obtain the normalized optimized Q-table; Based on the online initial link cost matrix and the normalized optimized Q table, the reshaped online initial link cost matrix is ​​calculated using the reshaping formula.

5. The routing determination method for underwater acoustic sensor networks according to claim 1, characterized in that, The expression for the reshaping formula is: ； in, The first in the reshaped online initial link cost matrix The node and the first The cost of online links between nodes; The first in the online initial link cost matrix The node and the first Online link cost between nodes; This is a truncation function; The prior strength coefficient; For the optimized Q-table after normalization, the first... The node and the first Q-values ​​between nodes; This is the lower bound of the scaling factor; This is the upper limit of the scaling factor.

6. The routing determination method for underwater acoustic sensor networks according to claim 1, characterized in that, Obtain the online initial link cost matrix and online initial adjacency matrix of the underwater acoustic sensor network, specifically including: Acquire all online underwater nodes of the underwater acoustic sensor network and the online network performance indicators between each online underwater node; the online network performance indicators include online latency, online power consumption, and online packet loss rate; the online underwater nodes include: online source nodes, online relay nodes, and online destination nodes; Based on all online underwater nodes and the performance indicators of the online network, a bidirectional weighted graph is constructed; Based on the bidirectional weighted graph, the online initial link cost matrix and the online initial adjacency matrix are determined.

7. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the routing determination method for an underwater acoustic sensor network according to any one of claims 1-6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the routing determination method for an underwater acoustic sensor network as described in any one of claims 1-6.

Images