Underwater routing protocol design method based on link awareness

Abstract

The invention discloses an underwater routing protocol design method based on link sensing, and belongs to the technical field of underwater wireless sensor communication networking. The invention aims to solve the problem that the link quality of the existing underwater routing protocol cannot ensure reliable transmission of data. Comprising the following steps: dividing underwater wireless sensors into clusters based on set roles in a node control data packet; each cluster comprises a master node and a corresponding slave node; each master node receives a sensing data packet of the corresponding slave node; each master node maintains attribute information of the adjacent master nodes based on the master node attribute data packets of the adjacent master nodes, and establishes a reward function for transmitting the to-be-transmitted data packets of the master nodes between the two master nodes based on the attribute information of the master nodes; and after a next-hop master node of the underwater route is determined according to the reward function value, transmitting the to-be-transmitted data packet of the current master node to the next-hop master node until data transmission from the to-be-transmitted data packets of all the master nodes to the water surface ship receiving terminal is completed. The underwater sensing data transmission device is used for transmitting underwater sensing data.

Description

Technical Field

[0001] The invention relates to a link-aware underwater routing protocol design method, and belongs to the technical field of underwater wireless sensor communication networking. Background Art

[0002] Underwater wireless sensor networks (UWSNs) are important communication facilities supporting underwater monitoring applications. However, the inherent high bit error rate, significant transmission delay and various interference factors of underwater acoustic channels, coupled with the high energy consumption and energy supply challenges faced by network nodes, increase the necessity and urgency of designing efficient routing protocols.

[0003] In recent years, with the rapid development of artificial intelligence technology, reinforcement learning, especially Q-learning algorithm, has become an important tool for solving complex decision-making problems. With its adaptability and trial-and-error learning mechanism, this algorithm allows the agent to dynamically adjust its action strategy based on real-time environmental feedback without relying on extensive prior knowledge. It has been introduced to solve the problem of underwater wireless sensor routing protocol. However, most of the existing underwater routing protocols combined with Q-learning are designed in an idealized environment, focusing mainly on optimizing energy consumption and reducing end-to-end delay, and often ignoring the significant impact of link quality on the success rate of data transmission. Due to the inherent high bit error rate and frequent interference in the underwater environment, in some application scenarios with extremely high requirements for data transmission reliability, frequent packet loss may cause a large amount of data to be retransmitted, which not only increases energy consumption but also makes it difficult to meet the application requirements of high reliability. Therefore, the development of an underwater wireless sensor network routing protocol with strong robustness, high reliability and optimized energy efficiency has become a key issue that needs to be solved urgently. Summary of the invention

[0004] Aiming at the problem that the link quality of the existing underwater routing protocol cannot guarantee reliable data transmission, the present invention provides a method for designing an underwater routing protocol based on link perception.

[0005] A link-aware underwater routing protocol design method of the present invention comprises:

[0006] Deploy an underwater wireless sensor network and divide the underwater wireless sensors into clusters based on the set roles in the node control data packet; each cluster includes a master node and corresponding slave nodes;

[0007] Each master node is regarded as an independent intelligent entity to receive the sensor data packets of the corresponding slave node; the routing forwarding data packets of each master node include the master node attribute data packets and the master node sensor data packets; each master node integrates all received slave node sensor data packets and the current master node sensor data packets into the master node to-be-delivered data packets; at the same time, each master node maintains the attribute information of the adjacent master node based on the master node attribute data packets of the adjacent master node, and establishes a reward function for the transmission of the master node to-be-delivered data packets between the two master nodes based on the attribute information of the master node; after determining the next hop master node of the underwater route according to the reward function value, the current master node to-be-delivered data packet is transmitted to the next hop master node until the data transmission of the master node to-be-delivered data packets of all master nodes to the surface ship receiving terminal is completed.

[0008] According to the link-aware underwater routing protocol design method of the present invention, in each cluster, the master node and the slave node complete the cluster division through the interaction of node control data packets; the master node sends cluster invitation information by broadcasting, and the slave node sends confirmation reply information to the master node after receiving the invitation information, thereby realizing the establishment of the cluster.

[0009] According to the link-aware underwater routing protocol design method of the present invention, the attribute information in the master node attribute data packet includes the current remaining energy E of the master node. pre 、The initial energy of the master node E ini , the average energy of all adjacent master nodes The depth difference D between the main node and the surface ship k , the maximum depth difference between the main node and the surface ship D max,k and the transmission distance d, where k represents the diffusion coefficient.

[0010] According to the link-aware underwater routing protocol design method of the present invention, the reward function is established as follows: Indicates that the master node i transmits the data packet a to be transmitted by the master node i Direct reward from neighboring master node j:

[0011]

[0012] Where χ 1 is the energy weight factor, χ 2 is the depth weight factor, χ 3 is the link quality weight factor, E is the energy factor, D is the depth factor, and J is the link quality factor;

[0013]

[0014] Where SNR is the average signal-to-noise ratio. When the transmission distance from master node i to adjacent master node j is d:

[0015]

[0016] Where E b It represents the average energy consumed to transmit one bit of data, A(d,f) is the path loss, f is the transmission frequency, α(f) is the absorption coefficient, A(d,f)=d k α(f) d , N 0 is the noise power spectral density under additive white Gaussian noise conditions.

[0017] According to the link-aware underwater routing protocol design method of the present invention,

[0018] According to the link-aware underwater routing protocol design method of the present invention, assuming that there are N master nodes in total, and each master node i has N-1 adjacent master nodes, the reward function Establish the N×N reward matrix R(i) corresponding to master node i:

[0019]

[0020] The value of the reward matrix R(i) is represented as Q, then Q K+1 (i,a j ) represents the (i, j)th element value of the reward matrix R(i) in the K+1th iteration, j = 1, 2, 3, ..., N, i = 1, 2, 3, ..., N;

[0021]

[0022] Where ξ∈(0,1] is the learning rate, γ∈(0,1] is the discount factor, and v K (j) is the maximum value of the jth row in the Kth iteration of the reward matrix R(i), select Q K+1 (i,a j )The adjacent master node corresponding to the maximum value is used as the next-hop master node.

[0023] According to the link-aware underwater routing protocol design method of the present invention,

[0024] In the formula represents the probability of success from master node i to adjacent master node j, which is set to 1; represents the probability of failure from primary node i to adjacent primary node j, which is set to 0; Data packet a to be transmitted by the master node i The reward function that is not transmitted externally at master node i.

[0025] According to the link-aware underwater routing protocol design method of the present invention, the information in the master node attribute data packet includes:

[0026] Source node address: indicates the source node ID of the current master node;

[0027] Destination address: indicates the address of the surface ship;

[0028] Next hop master node address;

[0029] Reward function value: records the Q of all elements in the reward matrix R(i) K+1 (i,a j ) value, and select the maximum Q K+1 (i,a j )value;

[0030] As well as the master node initial energy, the master node current remaining energy, the master node location information and the master node sensor data packet ID.

[0031] According to the link-aware underwater routing protocol design method of the present invention, the fields included in the node control data packet are:

[0032] Identifier: used to identify the type of node control data packet;

[0033] Node ID: information used to indicate the identity of the node;

[0034] Data: The control node controls the instructions or control information carried in the data packet and coordinates the clustering and roles of the nodes.

[0035] According to the link-aware underwater routing protocol design method of the present invention, the Q value of the reward matrix R(i) is calculated based on the Q-learning algorithm.

[0036] Beneficial effects of the present invention: During the network startup phase, the method of the present invention is divided into master nodes and slave nodes according to preset roles to optimize communication efficiency and reduce data packet collisions. After the node is deployed underwater, the configuration of the network role is completed by automatically identifying the information in the control data packet. The master node is responsible for inviting the slave node to join the cluster and completing the formation of the cluster. After that, the communication phase begins, the slave node sends the collected data to the master node, and then the master node transmits the data to the surface ship. Due to the inherent high bit error rate and frequent interference in the underwater environment, and the long distance between the master node and the surface ship, in order to ensure the effective transmission of the data packet, the reward function constructed by the present invention is equipped with a Q-learning algorithm to construct a transmission path, which comprehensively considers the link quality, residual energy and depth difference between nodes, improves the rationality and adaptability of selecting the next hop node, and designs the routing forwarding data packet format and the control data packet format.

[0037] The present invention significantly improves the data packet delivery rate. It has been verified that the performance is improved by 28% compared with the SEP protocol and nearly twice that of the LEACH protocol. The present invention better meets the requirements of underwater applications for network data transmission reliability, realizes efficient packet forwarding, and effectively improves energy utilization efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flow chart of the link-aware underwater routing protocol design method of the present invention;

[0039] Figure 2 It is a scene graph of the interaction between the agent and the environment;

[0040] Figure 3 This is a schematic diagram of the routing and forwarding data packet format of the master node;

[0041] Figure 4 It is a schematic diagram of the node control data packet format;

[0042] Figure 5 is a comparison chart of the number of surviving master nodes in different protocol transmission methods in the embodiment;

[0043] Figure 6 is based on Figure 5 Masternode mortality comparison chart;

[0044] Figure 7 This is a comparison chart of the amount of data received by surface ships using different protocol transmission methods. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0047] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0048] Specific implementation method 1. Combination Figure 1 As shown, the present invention provides a method for designing an underwater routing protocol based on link perception, comprising:

[0049] Deploy an underwater wireless sensor network and divide the underwater wireless sensors into clusters based on the set roles in the node control data packet; each cluster includes a master node and corresponding slave nodes;

[0050] Each master node is regarded as an independent intelligent entity to receive the sensor data packets of the corresponding slave node; the routing forwarding data packets of each master node include the master node attribute data packets and the master node sensor data packets; each master node integrates all received slave node sensor data packets and the current master node sensor data packets into the master node to-be-delivered data packets; at the same time, each master node maintains the attribute information of the adjacent master node based on the master node attribute data packets of the adjacent master node, and establishes a reward function for the transmission of the master node to-be-delivered data packets between the two master nodes based on the attribute information of the master node; after determining the next hop master node of the underwater route according to the reward function value, the current master node to-be-delivered data packet is transmitted to the next hop master node until the data transmission of the master node to-be-delivered data packets of all master nodes to the surface ship receiving terminal is completed.

[0051] As an example, in the initial deployment stage of the network, each node is assigned as a master node or a slave node according to a pre-set role strategy, and this role information is encapsulated in a control data packet. These nodes are then deployed to an underwater environment. After activation, they automatically identify the data in the control data packet (where 0 represents the master node and 1 represents the slave node) to configure their network roles. The master node and the slave node complete the cluster division through the interaction of the node control data packet. Once the role configuration is completed, in each cluster, the master node initiates the cluster formation process by broadcasting the cluster invitation information, and the invitation information is received by all the slave nodes within the communication range of the master node; after receiving the invitation information, the slave node sends a confirmation reply message to the master node. When the master node receives the confirmation reply from all the slave nodes, the clustering stage of the network is officially completed, thereby realizing the establishment of the cluster.

[0052] Combination Figure 2 As shown, during the operation of the network, the intelligent agent continuously maintains the relevant information of the neighboring nodes by parsing the received data packets forwarded by the routing. The attribute information in the master node attribute data packet includes the current remaining energy E of the master node. pre 、The initial energy of the master node E ini , the average energy E of all adjacent master nodes, the depth difference D between the master node and the surface ship k , the maximum depth difference between the main node and the surface ship D max,k and the transmission distance d, where k represents the diffusion coefficient.

[0053] In this implementation, after selecting a suitable hydrological condition area, the underwater sensor nodes are precisely deployed to ensure the optimal location of the nodes and the network coverage, thereby completing the deployment of the underwater wireless sensor network. After the nodes are deployed, the system will automatically generate a control data packet. Through the interactive mechanism of the control data packet, the network nodes automatically complete the cluster division according to the preset rules and parameters; after the cluster is formed, each master node is regarded as an independent intelligent agent in the communication stage. Each intelligent agent designs a corresponding reward function based on the interactive information with the surrounding environment, and dynamically adjusts its action strategy through real-time environmental feedback, thereby continuously optimizing the data transmission path; when designing the reward function, key factors such as the link quality between nodes, the residual energy of the nodes, and the depth difference are comprehensively considered to ensure the rationality and efficiency of the transmission decision; in order to avoid data conflicts during routing transmission, this implementation clearly stipulates the format of the routing forwarding data packet and introduces the control data packet. In the initial stage of network construction, the control data packet is used to coordinate the cluster formation process.

[0054] Furthermore, the design of the reward function is crucial to the performance of the routing protocol. In this embodiment, the link quality, residual energy, and depth difference are incorporated into the design of the reward function; the reward function is established based on the information in the master node attribute data packet: Indicates that the master node i transmits the data packet a to be transmitted by the master node i Direct reward from neighboring master node j:

[0055]

[0056] Where χ 1 is the energy weight factor, χ 2 is the depth weight factor, χ 3 is the link quality weight factor, E is the energy factor, D is the depth factor, and J is the link quality factor;

[0057]

[0058] Where SNR is the average signal-to-noise ratio (SNR). When the transmission distance from master node i to adjacent master node j is d:

[0059]

[0060] In this implementation, the Thorp model is used to describe underwater sound wave propagation;

[0061] Where E b It represents the average energy consumed to transmit one bit of data, A(d,f) is the path loss, f is the transmission frequency, α(f) is the absorption coefficient, A(d,f)=d k α(f)d , N 0 is the noise power spectral density under additive white Gaussian noise (AWGN) conditions.

[0062] In this embodiment,

[0063] Furthermore, assuming that there are N master nodes in total, and each master node i has N-1 adjacent master nodes, then the reward function Establish the N×N reward matrix R(i) corresponding to master node i:

[0064]

[0065] The value of the reward matrix R(i) is represented as Q, then Q K+1 (i,a j ) represents the (i, j)th element value of the reward matrix R(i) in the K+1th iteration, j = 1, 2, 3, ..., N, i = 1, 2, 3, ..., N;

[0066] The elements in the Q value of the reward matrix R(i) are updated by the following iterative formula:

[0067]

[0068] Where ξ∈(0,1] is the learning rate, which affects the update speed of the Q matrix, γ∈(0,1] is the discount factor, which means that the current action will have a greater impact on the current value than the future action, and v K (j) is the maximum value of the jth row in the Kth iteration of the reward matrix R(i). In order to ensure that the master node i can obtain the maximum reward when selecting the next hop node, the transmission path that maximizes the Q value will be adopted. After iterating the neighbor master nodes, Q K+1 (i,a j )The adjacent master node corresponding to the maximum value is used as the next-hop master node.

[0069] The Q value of each sensor node is updated through distributed computing methods. When the number of data packets cached locally by the source node reaches a certain amount, the best next-hop routing node is determined based on the current Q value;

[0070]

[0071] In the formula represents the probability of success from master node i to adjacent master node j, which is set to 1; represents the probability of failure from primary node i to adjacent primary node j, which is set to 0; Data packet a to be transmitted by the master node i The reward function that is not transmitted externally at master node i.

[0072] The N×N transition probability matrix P(i) can be defined as:

[0073]

[0074] Combination Figure 3 As shown, the information in the master node attribute data packet includes:

[0075] Source node address: indicates the source node ID of the current master node;

[0076] Destination address: the target node address to which the data packet should be sent, that is, the address of the surface ship;

[0077] Next hop master node address: the node address selected as the next hop during the transmission of the data packet;

[0078] Reward function value: The current node calculates the Q value of the neighboring node and records the Q value of all elements in the reward matrix R(i) K+1 (i,a j ) value, and select the maximum Q K+1 (i,a j ) value becomes the next hop;

[0079] Master node initial energy: the initial energy of the neighbor master node, which is convenient for measuring the energy consumption of the master node;

[0080] Current remaining energy of the master node: the remaining energy of the current neighbor master node, used to evaluate the continuous working ability of the master node;

[0081] Master node location information: Geographical location information of the current neighboring master nodes, used to determine the master node's location in the network and its relative position to other master nodes;

[0082] Master node sensor data packet ID: a unique identifier for each data packet, ensuring the uniqueness of the data packet in the network and avoiding duplicate transmission;

[0083] The master node sensing data packet includes data, that is, the actual data content to be transmitted.

[0084] The master node currently forwarding the data will read Figure 3 The reward function is constructed based on the data in . The neighbor node with the largest Q value will be selected as the next hop node. The current master node will then forward the data packet to the node with the largest Q value and update the data packet according to the relevant information of the next hop node. If a suitable next hop node cannot be selected, the current master node will only update the local information and discard the data packet to avoid unnecessary energy consumption and data conflicts.

[0085] Combination Figure 4As shown in the figure, the control data packet ensures that the network can correctly configure the node roles and communication structure at startup by transmitting control information to avoid conflicts during cluster construction. The fields contained in the node control data packet are:

[0086] Identifier: used to identify the type of node control data packet and determine whether it is a control data packet rather than a data packet;

[0087] Node ID: information used to indicate the identity of a node; the node ID that sends control packets can help the receiver identify the sender;

[0088] Data: The control node controls the instructions or control information carried in the data packet, coordinates the clustering and role allocation of nodes, or other network operations.

[0089] In this embodiment, the Q value of the reward matrix R(i) is calculated based on the Q-learning algorithm.

[0090] Example:

[0091] In the detection area of ​​5000m×5000m×5000m, 100 master nodes are evenly distributed, the coordinates of the surface ship are (2500, 2500, 5000), and the working frequency is set to 3kHz. The method of the present invention, the LEACH protocol and the SEP protocol are used for underwater routing respectively, and the following is obtained: Figure 5 and Figure 6 The master node survival and death data shown in FIG. 1 can be seen that the life cycles of the method of the present invention, the LEACH protocol and the SEP protocol are 189 rounds, 74 rounds and 123 rounds respectively. Figure 6 More intuitive presentation Figure 5 The data in show that the protocol proposed in the present invention performs well from the initial stage. With the continuous optimization of the Q-learning algorithm, even when a large number of master nodes fail in the later stage, this protocol makes the life cycle of the master node more stable and continuous by fully considering the link quality. Obviously, the protocol proposed in the present invention is significantly better than the other two protocols in terms of life cycle, showing higher energy efficiency performance.

[0092] Figure 7It is further shown that the number of data packets transmitted to the aggregation master node by the protocol proposed in the present invention, LEACH and SEP protocols are 1949, 1083 and 1521 respectively, indicating that the protocol proposed in the present invention far exceeds other protocols in terms of data transmission volume. Thanks to the longer life cycle, the amount of data transmitted by the protocol of the present invention is almost twice that of LEACH, and the transmission performance is improved by about 28% compared with the SEP protocol. As the number of master nodes gradually decreases during the operation of the protocol, the link quality of the channel becomes a key factor in determining whether the data packet can be successfully forwarded. By comprehensively considering the impact of link quality on data transmission, the present invention can maintain stable data transmission even when the number of master nodes decreases in the later stage, thereby achieving efficient and reliable packet forwarding.

[0093] In summary, the present invention fully considers the inherent high bit error rate and frequent interference characteristics of underwater environments, and incorporates link quality into the reward function design of the Q-learning algorithm, thereby improving the reliability of data transmission and the delivery success rate. The present invention designs a control data packet format to ensure that nodes can automatically configure roles and communication structures when the network starts, effectively avoid communication conflicts that may occur during cluster construction, and improve the initialization efficiency of the network. The present invention regards the master node as an intelligent agent and designs a special routing forwarding data packet format. Through the initial energy, residual energy and location information of neighbor nodes carried in the data packet, the intelligent agent can interact with the environment, accelerate the determination of routing paths, and optimize the data transmission process.

[0094] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that many modifications may be made to the exemplary embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in a manner different from that described in the original claims. It should also be understood that the features described in conjunction with a single embodiment may be used in other described embodiments.

Claims

1. A method for designing an underwater routing protocol based on link awareness, characterized in that: include: Deploy an underwater wireless sensor network and divide the underwater wireless sensors into clusters based on the roles set in the node control data packets; Each cluster includes a master node and corresponding slave nodes; Each master node is regarded as an independent intelligent entity to receive the sensor data packets of the corresponding slave node; the routing forwarding data packets of each master node include the master node attribute data packets and the master node sensor data packets; each master node integrates all received slave node sensor data packets and the current master node sensor data packets into the master node to-be-delivered data packets; at the same time, each master node maintains the attribute information of the adjacent master node based on the master node attribute data packets of the adjacent master node, and establishes a reward function for the transmission of the master node to-be-delivered data packets between the two master nodes based on the attribute information of the master node; after determining the next hop master node of the underwater route according to the reward function value, the current master node to-be-delivered data packet is transmitted to the next hop master node until the data transmission of the master node to-be-delivered data packets of all master nodes to the surface ship receiving terminal is completed.

2. The link-aware underwater routing protocol design method according to claim 1 is characterized in that: In each cluster, the master node and the slave node complete the cluster division through the interaction of node control data packets; the master node sends cluster invitation information through broadcasting, and the slave node sends confirmation reply information to the master node after receiving the invitation information, thereby realizing the establishment of the cluster.

3. The link-aware underwater routing protocol design method according to claim 1 is characterized in that: The attribute information in the master node attribute data packet includes the current remaining energy E of the master node pre 、The initial energy of the master node E ini , the average energy of all adjacent master nodes The depth difference D between the master node and the surface ship k , the maximum depth difference between the main node and the surface ship D max,k and the transmission distance d, where k represents the diffusion coefficient.

4. The link-aware underwater routing protocol design method according to claim 3 is characterized in that: The reward function is established as Indicates that the master node i transmits the data packet a to be transmitted by the master node i Direct reward from neighboring master node j: Where χ1 is the energy weight factor, χ2 is the depth weight factor, χ3 is the link quality weight factor, E is the energy factor, D is the depth factor, and J is the link quality factor; Where SNR is the average signal-to-noise ratio. When the transmission distance from master node i to adjacent master node j is d: Where E b It represents the average energy consumed to transmit one bit of data, A(d,f) is the path loss, f is the transmission frequency, α(f) is the absorption coefficient, A(d,f)=d k α(f) d , N0 is the noise power spectral density under additive white Gaussian noise conditions.

5. The link-aware underwater routing protocol design method according to claim 4 is characterized in that:

6. The link-aware underwater routing protocol design method according to claim 5 is characterized in that: Assuming there are N master nodes, each master node i has N-1 adjacent master nodes, then the reward function Establish the N×N reward matrix R(i) corresponding to master node i: The value of the reward matrix R(i) is represented as Q, then Q K+1 (i,a j ) represents the (i, j)th element value of the reward matrix R(i) in the K+1th iteration, j = 1, 2, 3, ..., N, i = 1, 2, 3, ..., N; Where ξ∈(0,1] is the learning rate, γ∈(0,1] is the discount factor, and v K (j) is the maximum value of the jth row in the Kth iteration of the reward matrix R(i), and Q is selected K+1 (i,a j )The adjacent master node corresponding to the maximum value is used as the next-hop master node.

7. The link-aware underwater routing protocol design method according to claim 6 is characterized in that: In the formula represents the probability of success from master node i to adjacent master node j, which is set to 1; represents the probability of failure from primary node i to adjacent primary node j, which is set to 0; Data packet a to be transmitted by the master node i The reward function that is not transmitted externally at master node i.

8. The link-aware underwater routing protocol design method according to claim 1 is characterized in that: The information in the master node attribute data packet includes: Source node address: indicates the source node ID of the current master node; Destination address: indicates the address of the surface ship; Next hop master node address; Reward function value: records the Q of all elements in the reward matrix R(i) K+1 (i,a j ) value, and select the maximum Q K+1 (i,a j )value; As well as the master node initial energy, the master node current remaining energy, the master node location information and the master node sensor data packet ID.

9. The link-aware underwater routing protocol design method according to claim 1, characterized in that: The fields contained in the node control packet are: Identifier: used to identify the type of node control data packet; Node ID: information used to indicate the identity of the node; Data: The control node controls the instructions or control information carried in the data packet and coordinates the clustering and roles of the nodes.

10. The link-aware underwater routing protocol design method according to claim 6, characterized in that: The Q value of the reward matrix R(i) is calculated based on the Q-learning algorithm.

Images