Energy-saving routing protocol, design method and system for underwater acoustic communication network based on power control

Abstract

The invention discloses a power control-based energy-saving routing protocol, a design method and a system for an underwater acoustic communication network, which assist in optimizing the routing protocol in combination with a physical layer design, select the transmitting power of an underwater acoustic sensor node through the priority of a data packet in a physical layer to carry out power control, dynamically select the transmitting power of the sensor node, and improve the reliability of the underwater acoustic communication network. An optimal routing strategy is made on the basis of a Q-Learning algorithm; in a network layer, a decision is made based on a distributed Q table, and the Q table is updated through the residual energy of the nodes, the depth of the nodes and the transmitting angle of the nodes, so that the routing strategy is optimized. An underwater transmitting node is regarded as an intelligent agent through a reinforcement learning mode, distributed training is carried out according to the state and feedback of the underwater transmitting node, and a routing decision is made independently; each underwater sensor node can automatically learn an optimal strategy in a complex environment, the utilization efficiency of network resources is improved, and the problems of overlarge energy consumption, low transmission efficiency and the like in a traditional method are solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of underwater acoustic communication sensor network routing communication protocol, and particularly relates to an underwater acoustic communication network power control energy-saving routing protocol, a design method and a system. BACKGROUND

[0002] In the underwater acoustic communication network, the communication between nodes faces various challenges, such as complex and changeable channel conditions, huge energy consumption, limited bandwidth and the like, which have important influences on the overall performance of the network, the node life and the transmission efficiency. Especially in the underwater acoustic communication environment, the instability of the channel and the energy limitation of the node require that the efficient use of resources must be fully considered when designing the communication protocol.

[0003] The power allocation technology can reduce the network energy consumption by reasonably adjusting the node transmission power, so that the network can run for as long as possible, and at the same time, the reliability of data transmission is ensured while saving energy. Therefore, the use of power control technology helps to effectively reduce the network energy consumption and improve the system performance.

[0004] The efficient routing strategy establishes one or more paths between the source node and the destination node through path selection and packet forwarding, selects an optimal path, and transmits data packets from the source node to the destination node hop by hop along the optimal path. A reasonable routing strategy can improve the network energy utilization efficiency.

[0005] On the one hand, the existing underwater acoustic communication strategy usually uses the same transmission power for each node, but still uses the maximum transmission power when the transmission distance is short, which causes energy waste. On the other hand, when the total network energy consumption is reduced by power control, the number of data packet forwarding hops is increased, which usually leads to an increase in the total distance of data packet forwarding, resulting in an increase in end-to-end delay.

[0006] Therefore, designing an efficient network routing strategy has become a research hotspot in underwater acoustic communication networks. SUMMARY

[0007] The technical problem to be solved by the application is to provide an underwater acoustic communication network power control energy-saving routing protocol, a design method and a system, which solve the problems of energy waste and low transmission rate caused by the use of the same power by each node in the prior art.

[0008] The application adopts the following technical solutions to solve the above technical problems:

[0009] The underwater acoustic communication network is based on a power-controlled energy-saving routing protocol. It treats underwater transmitting nodes as intelligent agents and achieves cross-layer optimization by combining the physical layer and the network layer. At the physical layer, the transmission power of the underwater acoustic sensor node is dynamically selected based on the priority of data packets. At the network layer, decisions are made based on a distributed Q-table and the Q-table is updated by considering the node's remaining energy, node depth, and node transmission angle to optimize the routing strategy.

[0010] A design method for an underwater acoustic communication network based on a power control energy-saving routing protocol, wherein the underwater acoustic communication network includes an intelligent agent system composed of several underwater transmitting nodes, and the specific design method includes the following steps:

[0011] Step 1: Initialize the network. Each underwater transmitter discovers its neighboring nodes, establishes a neighbor table, and establishes its own fixed Q table, setting initial values.

[0012] Step 2: During the node forwarding process, a temporary Q table is established to divide the data packets into high priority and low priority. The relay node constructs a candidate set and selects the corresponding Q value from the fixed Q table based on the nodes in the candidate set to construct a temporary Q table.

[0013] Step 3: Select the next-hop routing node. When the relay node generates a random number after building the temporary Q table, it selects a neighbor node from its candidate set as the next-hop routing node.

[0014] Step 4: Data forwarding and node temporary Q-table update and destruction, fixed Q-table update, the node adds the ID of the next hop node to the data packet, and selects the appropriate method for data forwarding according to the different transmit power of the node, and performs the corresponding operation on the Q-table;

[0015] Step 5: Repeat steps 2 to 4 to pre-train the network and obtain a converged fixed Q-table for each node.

[0016] Step 6: After network convergence, each node forwards the data packets sequentially until the data packets are sent to the aggregation node.

[0017] The specific process of step 1 is as follows:

[0018] Step 1.1: The aggregation node sends Hello packets using high power. When a relay node within the aggregation node's transmission range receives the Hello packets, it determines whether the aggregation node meets the criteria for neighboring nodes. If it does, the relay node adds the aggregation node to its neighbor table and marks it as 1. At the same time, the relay node continues to forward Hello packets using high power.

[0019] Step 1.2: At intervals, the aggregation node sends Hello packets using low power. Relay nodes within the aggregation node's transmission range receive the Hello packets and update their markers to 0. Meanwhile, the relay nodes continue to send Hello packets using low power.

[0020] Step 1.3: Repeat steps 1.1 and 1.2 until the neighbor tables of all nodes are established. At the same time, initialize the fixed Q table of each node.

[0021] In step 1.3, a fixed Q table for each node is initialized. The row value is the ID number of the relay node itself, and the column value is the Q value corresponding to the node in the relay node candidate set. The Q value is initialized to 0.

[0022] In step 2, after dividing the data packets into high-priority and low-priority packets, for high-priority data packets, the relay node adds all neighboring nodes to its candidate set; for low-priority data packets, the relay node first adds the nodes marked as 0 in the neighbor table to the candidate set. If the candidate set is empty at this time, it continues to add the nodes marked as 1 in the neighbor table to the candidate set until the candidate set is completed. Then, the relay node selects the corresponding Q value from the fixed Q table based on the nodes in the candidate set and constructs a temporary Q table.

[0023] The rule for selecting the next-hop routing node in step 3 is as follows: if the generated random number is greater than or equal to the selection factor, the relay node randomly selects a neighboring node from its candidate set as the next-hop routing node; if the generated random number is less than the selection factor, the node corresponding to the largest Q value in the temporary Q table is selected as the next-hop routing node.

[0024] In step 4, the node adds the ID of the next-hop node to the data packet and performs the following operations based on the node's different transmit power:

[0025] When the transmitting node's transmit power is high, all neighboring nodes will receive the data packet and identify the next-hop ID in the data packet. If the neighboring node's own ID is the same as the next-hop ID in the data packet, it will continue to forward the data packet and send an ACK-1 packet back to the transmitting node. After the transmitting node extracts the relevant information from the ACK-1 packet, it sets the HD in the ACK-1 packet to 0 and ends the communication. At the same time, it updates its own temporary Q table. If the transmitting node does not receive the ACK-1 packet, it will discard the data packet.

[0026] When the transmitting node's transmit power is low, the following steps are performed:

[0027] Step a: If the neighboring node's own ID is the same as the next-hop ID in the data packet, it continues to forward the data packet as a relay node;

[0028] Step b: Calculate and temporarily store the relative angle between the sending node and the relay node, and simultaneously send an ACK-1 packet back to the sending node;

[0029] Step c: After the relay node forwards the data packet, it executes step a, calculates and temporarily stores the relative angle between the relay node and the next relay node, and the next relay node sends back an ACK-1 packet to the relay node.

[0030] Step d: Calculate the forwarding angle of the relay node based on the angle between the sending node and the relay node, and the angle between the relay node and the next relay node. At the same time, the relay node sends an ACK-2 packet back to the sending node.

[0031] Step e: The sending node calculates the reward and updates its own temporary Q table based on the relevant information in the ACK-1 and ACK-2 packets;

[0032] After forwarding is complete, the sending node updates the Q value of the corresponding action in the temporary Q table to the fixed Q table and destroys the temporary Q table.

[0033] When both the sending node and the relay node use low-power transmission, the reward received by the sending node is adjusted.

[0034] The underwater acoustic communication network is a communication system based on a power control energy-saving routing protocol, which includes a surface receiver and several underwater transmitter nodes. Each underwater transmitter node is regarded as an intelligent agent. Each intelligent agent communicates according to the design method of the energy-saving routing protocol to realize information interaction with the surface receiver.

[0035] A computer-readable storage medium storing computer-readable instructions that, when executed by a processor, invoke the steps of the method.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] 1. Advantages of Power Control Optimization Based on Reinforcement Learning: Compared to existing technologies where all nodes typically have the same transmission power, but the maximum transmission power is still used even at short distances, resulting in energy waste, this invention assists in optimizing the routing protocol by combining the physical layer. Power control is performed at the physical layer to dynamically select the transmission power of sensor nodes, and the optimal routing strategy is determined based on the Q-Learning algorithm. This optimization strategy enables the agent to automatically learn the optimal strategy in complex multi-node environments, improving network resource utilization efficiency and solving problems such as excessive energy consumption and low transmission efficiency encountered in traditional methods, thereby improving resource utilization efficiency and overall system performance.

[0038] 2. This invention models the routing problem of underwater sensor networks as a Markov Decision Process (MDP). Utilizing a distributed Q-Learning algorithm, each underwater transmitter node is treated as an agent. Each node independently makes decisions based on its own Q-table, selecting the next-hop node, updating the Q-table, and finding the optimal routing strategy. Distributed training is implemented across nodes, eliminating the need for centralized information exchange and control, thereby reducing information interaction and improving network energy efficiency.

[0039] 3. By using a strategy of classifying data packets, this invention enables high-priority data to be sent at high power and low-priority data packets to be sent at low power, thus avoiding a significant increase in end-to-end latency. Attached Figure Description

[0040] Figure 1 This is a data forwarding flowchart of the underwater acoustic communication network based on the power control energy-saving routing protocol method of the present invention.

[0041] Figure 2 This is a diagram of the intelligent agent data transmission model of the present invention.

[0042] Figure 3 This is a comparison chart of the energy consumption of the method proposed in this invention and other existing methods.

[0043] Figure 4 This is a comparison chart of the average delay between the method proposed in this invention and other existing methods. Detailed Implementation

[0044] The structure and working process of the present invention will be further described below with reference to the accompanying drawings.

[0045] To further optimize network energy efficiency and avoid energy waste, this scheme jointly optimizes and improves routing strategies at the physical and network layers, proposing a power-controlled energy-saving routing protocol and its design method to enhance network energy efficiency. This invention combines physical layer design to assist in optimizing the routing protocol. At the physical layer, power control is achieved by dynamically selecting the transmission power of underwater acoustic sensor nodes based on data packet priority, and the optimal routing strategy is made based on the Q-Learning algorithm. At the network layer, decisions are made based on a distributed Q-table, and the Q-table is updated using the node's remaining energy, node depth, and transmission angle to optimize the routing strategy. Through reinforcement learning, underwater transmitting nodes are treated as agents, undergoing distributed training based on their own states and feedback to independently make routing decisions. Each underwater sensor node can automatically learn the optimal strategy in complex environments, thereby improving network resource utilization efficiency and solving problems such as excessive energy consumption and low transmission rates encountered in traditional methods.

[0046] The underwater acoustic communication network is based on a power-controlled energy-saving routing protocol. It treats underwater transmitting nodes as intelligent agents and achieves cross-layer optimization by combining the physical layer and the network layer. At the physical layer, the transmission power of the underwater acoustic sensor nodes is dynamically adjusted by the priority of data packets. At the network layer, decisions are made based on a distributed Q-table and the Q-table is updated by the remaining energy of the node, the depth of the node, and the transmission angle of the node, so as to optimize the routing strategy.

[0047] A design method for an underwater acoustic communication network based on a power control energy-saving routing protocol, wherein the underwater acoustic communication network includes an intelligent agent system composed of several underwater transmitting nodes, and the specific design method includes the following steps:

[0048] Step 1: Initialize the network. Each underwater transmitter discovers its neighboring nodes, establishes a neighbor table, and establishes its own fixed Q table, setting initial values.

[0049] Step 2: During the node forwarding process, a temporary Q table is established to divide the data packets into high priority and low priority. The relay node constructs a candidate set and selects the corresponding Q value from the fixed Q table based on the nodes in the candidate set to construct a temporary Q table.

[0050] Step 3: Select the next-hop routing node. When the relay node generates a random number after building the temporary Q table, it selects a neighbor node from its candidate set as the next-hop routing node.

[0051] Step 4: Data forwarding and node temporary Q-table update and destruction, fixed Q-table update, the node adds the ID of the next hop node to the data packet, and selects the appropriate method for data forwarding according to the different transmit power of the node, and performs the corresponding operation on the Q-table;

[0052] Step 5: Repeat steps 2 to 4 to pre-train the network and obtain a converged fixed Q-table for each node.

[0053] Step 6: After network convergence, each node forwards the data packets sequentially until the data packets are sent to the aggregation node.

[0054] Specific embodiments, such as Figures 1 to 4 As shown:

[0055] The design method of underwater acoustic communication network based on power control energy-saving routing protocol includes the following steps:

[0056] Step 1: Initialize the network. Each node in the underwater sensor network discovers its neighboring nodes and establishes a fixed Q-table. This system establishes the actions, states, and reward functions of each underwater node based on the MDP model, and uses a Q-learning strategy to find the optimal transmission path that balances node energy consumption and latency. When the system is initially deployed, the aggregation node... First, use high power. Send Hello packets, high power Set to 2 watts, the Hello packet data format includes 13 bits of data, namely the packet type (2 bits), packet ID (10 bits), and transmit power indication value (…). (1 bit). If node At the aggregation node Within the transmission range, and The criteria for determining neighboring nodes are met. Will Add to neighbor table In this context, the neighbor table is defined as follows: , and marked as ,at the same time Continue to use high power Forward the Hello packet. After a period of time, the aggregation node... Use low power Send Hello packets, low power Set to 1 watt, if the node At the aggregation node Within the transmission range, It will receive the Hello packet and update it. .at the same time Continue using low power Send Hello packets to the underwater nodes, continuing this process until the neighbor tables of all nodes are established. During this process, initialize a fixed Q-table for each node with an initial value of 0; its row value is the node's... Its own ID number, the column value is the node Candidate set The Q-value corresponding to the node in the diagram.

[0057] In this step, the time interval is determined according to the specific circumstances and does not need to be specified. Those skilled in the art can set it themselves based on the number and depth of network nodes.

[0058] Step 2: Establishment of a temporary Q-table during node forwarding. The source node divides data packets into high-priority and low-priority categories. For high-priority data packets ( ),node Add all neighboring nodes to the candidate set of its next-hop forwarding nodes. Among them. And for low-priority data packets ( ),node First, check the neighbor table. The node is added to its candidate set. If the candidate set is empty at this point, continue adding to the neighbor table. Nodes to candidate set In the middle. When Next-hop forwarding node candidate set After construction is complete, based on the node IDs in the candidate set, the corresponding Q values ​​are selected from its own fixed Q table to construct a temporary Q table. The row values ​​in the temporary Q table are the nodes. Its own ID number, the column value is the node Candidate set The Q-value corresponding to the node in the diagram.

[0059] Step 3: Select the next-hop routing node. After constructing the temporary Q-table, random numbers are generated. Meanwhile, the selection factor is defined:

[0060] (1)

[0061] in It is a constant. Total training time This represents the current moment in network operation. When... At that time, node From its candidate set Randomly select a node as the next hop. In this case, the node corresponding to the largest Q value in the temporary Q table is selected. As the next-hop route, the data format of the forwarded packets includes packet type (2 bits), packet ID (10 bits), packet priority (1 bit), transmit power identifier (1 bit), next-hop node ID (10 bits), current node depth (16 bits), and data (~ bits).

[0062] Step 4: Data forwarding and updating / destroying temporary Q-tables for nodes, and updating fixed Q-tables. Nodes Will The ID is added to the data packet to be forwarded when Its transmission power is high power hour, All neighboring nodes will receive the data packet and identify the next-hop ID in the data packet. If the neighboring node's own ID is the same as the next-hop ID in the data packet... Continue forwarding data packets and to The ACK-1 packet is sent as feedback. The ACK-1 packet data format consists of 64 bits, including the packet type (2 bits), the current node ID (…), and other information. 10 bits), current node depth ( 16 bits), current node energy ( 20 bits), maximum Q value ( 14 bits), hole node flag ( The ACK-1 packet contains the instruction (1 bit) and the wait bit (Hd, 1 bit). At this point, the ACK-1 packet is sent back with the instruction... The sending node is notified to end communication after receiving ACK-1. Upon receiving an ACK-1 packet, update its own temporary Q table; if it cannot receive an ACK-1 packet, discard the data packet directly.

[0063] when The transmission power is low power. When a neighboring node's own ID is the same as the next-hop ID in the data packet, the next-hop node... Upon receiving a data packet, it will continue to forward the data packet and perform calculations. and relative angle And temporarily save it, and give feedback to One ACK-1 packet, simultaneously ,at this time Will wait The ACK-2 packet is pending. The node continues to forward data packets and receives them. Next hop node After receiving the ACK-1 packet, calculate and relative angle And combine the previously stored angles The node forwarding angle is calculated. Subsequently Towards Continuing with the feedback, an ACK-2 packet is sent. The ACK-2 packet contains 20 bits of data, including the packet type (2 bits), the current node ID (...). 10 bits) and node forwarding angle ( (8 bits) Node Extract from The reward value is calculated based on the relevant information in the ACK-1 and ACK-2 packets. The reward value is defined as follows:

[0064] (2)

[0065] in, Indicates time Time node The remaining energy, Indicates time Time node depth, Indicates time Time node The angle from which it is forwarded.

[0066] When node and nodes When all nodes use low power to send data packets, The obtained rewards are adjusted as follows:

[0067] (3)

[0068] in Represents a node With nodes The depth difference for The next hop forwarding node.

[0069] The temporary Q-table is updated according to the following formula, and the Q-value is calculated as follows:

[0070] (4)

[0071] Among them, discount rate , For learning rate, , , They are respectively The state, actions, and rewards at any given moment This indicates the state at the next time step. For each time step... Action definition Current node or Data is sent to the next-hop node via unicast communication. Current node The state is defined as , The next state is , yes The selected next-hop node. The definition is shown in equations (2) and (3).

[0072] node After forwarding is complete, update the Q value of the corresponding action in the temporary Q table. The fixed Q table is then destroyed.

[0073] Step 5: Repeat steps 2 to 4 to pre-train the network and obtain a converged fixed Q-table for each node.

[0074] Step Six: Packet forwarding after stable network operation. A stable, fixed Q-table was obtained after the pre-training phase, and the nodes... When a data packet needs to be sent, the node A temporary Q-table will be built based on the candidate set, and the node with the largest Q-value will be selected from the temporary Q-table. Send data packets as the next-hop node. Similarly, All neighboring nodes within the transmission range will receive the data packet and identify the next-hop ID. If a neighboring node's own ID matches the next-hop ID in the data packet, It will continue to forward data packets, and according to High-power or low-power transmission power to The system will send back either an ACK-1 or ACK-2 packet; otherwise, the packet will be discarded. Then... The algorithm updates its temporary Q-table based on the information in the ACK-1 or ACK-2 packet, uses Q-Learning reinforcement learning, and evaluates the quality of actions based on Q-values. To make the algorithm make decisions that approach the optimal level, this paper uses the TD algorithm for training and updates the Q-values ​​using formula (4). Finally, the fixed Q-table is updated. Each node forwards the data packets to the aggregation node in sequence.

[0075] In summary, the key technical points that this method needs to protect are summarized as follows:

[0076] 1. Underwater acoustic communication network modeling: Each underwater transmitting node is regarded as an intelligent agent, and the underwater acoustic communication network topology is established by taking into account the multipath effect and signal attenuation characteristics of the underwater environment.

[0077] 2. Distributed Routing Protocol Based on Reinforcement Learning and Power Control: This approach combines the physical layer to assist in optimizing the routing protocol. Power control is implemented at the physical layer to dynamically select high or low transmission power for sensor nodes, and the optimal routing strategy is selected based on the Q-Learning algorithm. Through reinforcement learning, each node can automatically learn the optimal strategy in complex environments. Distributed training is achieved across nodes, eliminating the need for centralized information exchange and control. This reduces the amount of information interaction, improves network energy utilization efficiency, and solves the problems of excessive energy consumption and low transmission efficiency encountered in traditional methods.

[0078] 3. Reward Function Design: The reward function uses node energy and node depth as positive rewards, while considering node forwarding angle as a penalty. By balancing energy and depth through the weights of energy and depth, the number of multi-hop nodes can be minimized, and the energy consumption rate and average end-to-end latency can be reduced.

[0079] Based on the above method, this solution also proposes a communication system for underwater acoustic communication network based on a power control energy-saving routing protocol, including a surface receiver and several underwater transmitting nodes. Each underwater transmitting node is regarded as an intelligent agent, and each intelligent agent communicates according to the design method of the energy-saving routing protocol to realize information interaction with the surface receiver.

[0080] A computer-readable storage medium storing computer-readable instructions that, when executed by a processor, invoke the steps of the method.

[0081] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0082] It should be understood that this solution is not limited to the specific embodiments described above. Devices and structures not described in detail herein should be understood as being implemented in a manner common to the art. Any person skilled in the art can make many possible variations and modifications to this solution, or modify it into equivalent embodiments, without departing from the scope of this solution, using the methods and techniques disclosed above. This does not affect the substantive content of this solution. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of this solution, without departing from its scope, still fall within the protection scope of this solution.

Claims

1. An energy-saving routing protocol for underwater acoustic communication networks based on power control, characterized in that: The underwater transmitting node is regarded as an intelligent agent. It achieves cross-layer optimization by combining the physical layer and the network layer. At the physical layer, the transmitting power of the underwater acoustic sensor node is dynamically selected by the priority of the data packets. At the network layer, decisions are made based on the distributed Q-table, and the Q-table is updated by considering the remaining energy of the nodes, the depth of the nodes, and the emission angle of the nodes, thereby optimizing the routing strategy.

2. A design method for underwater acoustic communication networks based on power control energy-saving routing protocols, characterized by: The underwater acoustic communication network includes an intelligent agent system composed of several underwater transmitting nodes. The specific design method includes the following steps: Step 1: Initialize the network. Each underwater transmitter discovers its neighboring nodes, establishes a neighbor table, and establishes its own fixed Q table, setting initial values. Step 2: During the node forwarding process, a temporary Q table is established to divide the data packets into high priority and low priority. The relay node constructs a candidate set and selects the corresponding Q value from the fixed Q table based on the nodes in the candidate set to construct a temporary Q table. Step 3: Select the next-hop routing node. When the relay node generates a random number after building the temporary Q table, it selects a neighbor node from its candidate set as the next-hop routing node. Step 4: Data forwarding and node temporary Q-table update and destruction, fixed Q-table update, the node adds the ID of the next hop node to the data packet, and selects the appropriate method for data forwarding according to the different transmit power of the node, and performs the corresponding operation on the Q-table; Step 5: Repeat steps 2 to 4 to pre-train the network and obtain a converged fixed Q-table for each node. Step 6: After network convergence, each node forwards the data packets sequentially until the data packets are sent to the aggregation node.

3. The design method of the underwater acoustic communication network based on the power control energy-saving routing protocol according to claim 2, characterized in that: The specific process of step 1 is as follows: Step 1.1: The aggregation node sends Hello packets using high power. When a relay node within the aggregation node's transmission range receives the Hello packets, it determines whether the aggregation node meets the criteria for neighboring nodes. If it does, the relay node adds the aggregation node to its neighbor table and marks it as 1. At the same time, the relay node continues to forward Hello packets using high power. Step 1.2: At intervals, the aggregation node sends Hello packets using low power. Relay nodes within the aggregation node's transmission range receive the Hello packets and update their markers to 0. Meanwhile, the relay nodes continue to send Hello packets using low power. Step 1.3: Repeat steps 1.1 and 1.2 until the neighbor tables of all nodes are established. At the same time, initialize the fixed Q table of each node.

4. The design method of the underwater acoustic communication network based on the power control energy-saving routing protocol according to claim 3, characterized in that: In step 1.3, a fixed Q table for each node is initialized. The row value is the ID number of the relay node itself, and the column value is the Q value corresponding to the node in the relay node candidate set. The Q value is initialized to 0.

5. The design method of the underwater acoustic communication network based on the power control energy-saving routing protocol according to claim 4, characterized in that: In step 2, after dividing the data packets into high-priority and low-priority packets, for high-priority data packets, the relay node adds all neighboring nodes to its candidate set; for low-priority data packets, the relay node first adds the nodes marked as 0 in the neighbor table to the candidate set. If the candidate set is empty at this time, it continues to add the nodes marked as 1 in the neighbor table to the candidate set until the candidate set is completed. Then, the relay node selects the corresponding Q value from the fixed Q table based on the nodes in the candidate set and constructs a temporary Q table.

6. The design method of the underwater acoustic communication network based on the power control energy-saving routing protocol according to claim 5, characterized in that: The rule for selecting the next-hop routing node in step 3 is as follows: if the generated random number is greater than or equal to the selection factor, the relay node randomly selects a neighboring node from its candidate set as the next-hop routing node; if the generated random number is less than the selection factor, the node corresponding to the largest Q value in the temporary Q table is selected as the next-hop routing node.

7. The design method of the underwater acoustic communication network based on the power control energy-saving routing protocol according to claim 6, characterized in that: In step 4, the node adds the ID of the next-hop node to the data packet and performs the following operations based on the node's different transmit power: When the transmitting node's transmit power is high, all neighboring nodes will receive the data packet and identify the next-hop ID in the data packet. If the neighboring node's own ID is the same as the next-hop ID in the data packet, it will continue to forward the data packet and send an ACK-1 packet back to the transmitting node. After the transmitting node extracts the relevant information from the ACK-1 packet, it sets the HD in the ACK-1 packet to 0 and ends the communication. At the same time, it updates its own temporary Q table. If the transmitting node does not receive the ACK-1 packet, it will discard the data packet. When the transmitting node's transmit power is low, the following steps are performed: Step a: If the neighboring node's own ID is the same as the next-hop ID in the data packet, it continues to forward the data packet as a relay node; Step b: Calculate and temporarily store the relative angle between the sending node and the relay node, and simultaneously send an ACK-1 packet back to the sending node; Step c: After the relay node forwards the data packet, it executes step a, calculates and temporarily stores the relative angle between the relay node and the next relay node, and the next relay node sends back an ACK-1 packet to the relay node. Step d: Calculate the forwarding angle of the relay node based on the angle between the sending node and the relay node, and the angle between the relay node and the next relay node. At the same time, the relay node sends an ACK-2 packet back to the sending node. Step e: The sending node calculates the reward and updates its own temporary Q table based on the relevant information in the ACK-1 and ACK-2 packets; After forwarding is complete, the sending node updates the Q value of the corresponding action in the temporary Q table to the fixed Q table and destroys the temporary Q table.

8. The design method of the underwater acoustic communication network based on the power control energy-saving routing protocol according to claim 7, characterized in that: When both the sending node and the relay node use low-power transmission, the reward received by the sending node is adjusted.

9. A communication system for underwater acoustic communication networks based on a power control energy-saving routing protocol, characterized in that: It includes a surface receiver and several underwater transmitter nodes. Each underwater transmitter node is regarded as an intelligent agent. Each intelligent agent communicates according to the design method of the energy-saving routing protocol described in any one of claims 2 to 8 to realize information interaction with the surface receiver.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions that, when executed by a processor, invoke the steps of the method according to any one of claims 2 to 8.

Images