Constrained sensing graph-enhanced routing decision-making method and related equipment for underwater acoustic sensor networks

By acquiring local observation information from an underwater acoustic sensor network to generate a feasible next-hop set, and combining action masking and graph augmentation value assessment, a distributed routing decision model is constructed. This solves the problems of local greediness and energy imbalance in routing protocols in underwater networks, and improves the stability and energy utilization efficiency of the network.

CN123123230APending Publication Date: 2026-11-17SUN YAT SEN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611270538.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-20
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In underwater acoustic sensor networks, existing routing protocols suffer from problems such as local greedy decision-making, energy imbalance, and parameter adjustment relying on experience in complex dynamic environments, resulting in insufficient network lifetime and stability. Furthermore, existing learning-based routing methods lack explicit modeling of constraints such as delivery failure and latency exceeding limits, which can easily lead to imbalances between metrics and unstable training.

Method used

By acquiring local observation information of the current forwarding node, a set of feasible next hops is generated. Combined with action masking mechanism and graph augmentation value assessment, a distributed routing decision model is constructed, constrain-aware training is performed, and the target routing action is output to ensure that the routing decision is made within the feasible range.

Benefits of technology

It improves the routing stability and energy efficiency of underwater acoustic sensor networks, reduces unnecessary forwarding and energy consumption, and enhances the reliability and sustainability of the network in dynamic environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123230A_ABST
    Figure CN123123230A_ABST
Patent Text Reader

Abstract

This invention discloses a constraint-aware graph-enhanced routing decision-making method and related equipment for underwater acoustic sensor networks: acquiring local observation information of the current forwarding node; performing multi-dimensional feasibility assessment on candidate neighbor nodes to generate a feasible next-hop set; inputting the local observation information and the feasible next-hop set into a distributed routing decision-making model to be trained, and outputting pre-trained routing actions; executing the pre-trained routing actions to obtain execution results, and updating the local observation information; constructing a dynamic feasible graph using the updated local observation information, and further performing graph-enhanced value assessment to obtain state value estimation; constrain-aware training of the distributed routing decision-making model, deploying the trained distributed routing decision-making model to underwater nodes, inputting the local observation information and the feasible next-hop set, and outputting the target routing action. This invention can improve routing stability and energy utilization efficiency, and can be widely applied in the field of wireless sensor network technology.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of wireless sensor network technology, and in particular to a constrained sensing graph-enhanced routing decision method and related equipment for underwater acoustic sensor networks. Background Technology

[0002] Underwater acoustic sensor networks typically consist of multiple underwater sensor nodes, relay nodes, and surface or underwater convergence nodes, and are widely used in marine environmental monitoring, underwater target detection, marine engineering inspection, polar observation, marine ranching, port security, and marine resource development. Because electromagnetic waves attenuate significantly underwater, underwater nodes often exchange data via underwater acoustic communication. Underwater acoustic communication is characterized by low propagation speed, narrow bandwidth, significant channel fluctuations, long transmission time, and a high probability of bit errors and packet loss. Furthermore, underwater sensor nodes are usually battery-powered, resulting in high maintenance and battery replacement costs; node energy is a key factor affecting network lifespan and mission reliability.

[0003] Under the aforementioned constraints, data packets in underwater acoustic sensor networks are typically difficult to send directly from the source node to the sink node, and instead require hop-by-hop forwarding through multiple relay nodes. Therefore, routing strategies must not only ensure that data packets reach the sink node as successfully as possible, but also consider end-to-end latency, node energy consumption, link reliability, network load balancing, and routing stability. If the routing strategy excessively favors a few nodes that are close to the sink node or shallowly located, these nodes may experience rapid energy depletion, creating energy holes or routing holes, thereby affecting the overall network lifespan and continuous service capability.

[0004] Traditional underwater routing protocols typically rely on manually set local sorting rules or thresholds for next-hop selection. In complex and dynamic environments, these methods suffer from problems such as obvious local greedy decision-making, insufficient characterization of system-level constraints, prominent issues of voids and energy imbalances, and dependence on experience for parameter adjustment.

[0005] In recent years, reinforcement learning and deep reinforcement learning have been used for underwater network routing. However, existing learning-based routing methods often combine multiple metrics into a single reward, lacking explicit modeling of constraints such as delivery failures and latency exceeding limits, which can easily lead to imbalances among metrics. In a dynamically changing set of next hops, directly outputting the action probabilities of all nodes can easily result in the selection of unreachable nodes, leading to unstable training. Relying solely on local information for training value estimation is inaccurate, affecting convergence and generalization. When there is no next hop that satisfies constraints, there is a lack of a safe backoff mechanism, which can easily increase energy consumption and latency. Summary of the Invention

[0006] In view of this, the main objective of the embodiments of the present invention is to provide a constrained perception graph-enhanced routing decision method and related equipment for underwater acoustic sensor networks, in order to solve at least one of the problems of the prior art and improve routing stability and energy utilization efficiency.

[0007] To achieve the above objectives, one aspect of the present invention provides a constrained sensing graph-enhanced routing decision method for underwater acoustic sensor networks, comprising: Obtain local observation information of the current forwarding node; Based on the local observation information, a multi-dimensional feasibility judgment is made on the candidate neighbor nodes to generate a set of feasible next hops; The local observation information and the feasible next-hop set are input into the distributed routing decision model to be trained, and the pre-trained routing actions are output. The pre-trained routing action is executed to obtain the execution result, and the local observation information is updated based on the execution result; Using the updated local observation information, a dynamic feasible graph is constructed, and graph augmentation value assessment is performed on the dynamic feasible graph to obtain a state value estimate; Using the state value estimation and the execution result, the distributed routing decision model to be trained is subjected to constraint-aware training to obtain the trained distributed routing decision model. The trained distributed routing decision model is deployed to an underwater node. The local observation information to be decided and the set of feasible next hops to be decided are input into the trained distributed routing decision model, and the target routing action is output.

[0008] In some embodiments, the step of performing multi-dimensional feasibility assessment on candidate neighbor nodes based on the local observation information and generating a feasible next-hop set includes the following steps: Based on the local observation information, extract the first remaining energy of the current forwarding node, the second remaining energy of the candidate neighbor node, the estimated single-hop delay of the candidate link, and the access history in the cumulative state of the current routing process; Determine whether the candidate neighbor node is within the communication coverage area of ​​the current forwarding node to obtain a first determination result; Determine whether the first remaining energy is greater than or equal to a preset transmission energy threshold to obtain a second determination result; Determine whether the second remaining energy is greater than or equal to a preset received energy threshold to obtain a third determination result; Determine whether the estimated single-hop delay is less than the preset single-hop delay upper limit to obtain the fourth determination result; Determine whether the candidate neighbor node exists in the access history to obtain the fifth determination result; The feasible next hop set is generated based on the first judgment result, the second judgment result, the third judgment result, the fourth judgment result, and the fifth judgment result.

[0009] In some embodiments, inputting the local observation information and the feasible next-hop set into the distributed routing decision model to be trained, and outputting pre-trained routing actions, includes the following steps: When the feasible next hop set is not empty, an action mask is generated based on the feasible next hop set; The local observation information is input into the feature encoding unit of the distributed routing decision model to be trained, and a candidate action feature vector is output; each component in the candidate action feature vector corresponds to each of the candidate neighbor nodes. Based on the action mask, infeasible candidate nodes in the candidate action feature vector are masked to obtain an action score vector; The action score vector is normalized to obtain the selection probability distribution of each feasible candidate node in the feasible next hop set; The feasible candidate node with the highest probability in the selection probability distribution is selected as the next hop node to obtain the pre-trained routing action.

[0010] In some embodiments, the method further includes the following steps: When the set of feasible next hops is empty, activate the safe rollback action; In response to the security rollback action, the current routing process is terminated.

[0011] In some embodiments, the step of constructing a dynamic feasible graph using the updated local observation information, performing graph augmentation value assessment on the dynamic feasible graph, and obtaining a state value estimate includes the following steps: Extract the state features of underwater nodes and sink nodes from the updated local observation information, and extract the feasible forwarding link relationships between underwater nodes and between underwater nodes and sink nodes; Based on the underwater node and the convergence node, a graph node set is constructed, and based on the feasible forwarding link relationship, a directed edge set is constructed. Based on the graph node set and the directed edge set, the dynamic feasible graph is obtained. Based on the state characteristics, the feasible forwarding link relationships, the first indication information of the current forwarding node, and the second indication information of the aggregation node, a graph augmentation value assessment is performed on the dynamic feasible graph to obtain the state value estimate.

[0012] In some embodiments, the process of constrain-aware training of the distributed routing decision model to be trained using the state value estimation and the execution result to obtain the trained distributed routing decision model includes the following steps: Obtain the delivery failure cost, latency cost, and energy consumption cost from the execution result; Based on the first comparison result of the delivery failure cost and the first preset budget, the second comparison result of the delay cost and the second preset budget, and the third comparison result of the energy consumption cost and the third preset budget, the delivery reliability constraint corresponding to the delivery failure cost, the delay constraint corresponding to the delay cost, and the energy consumption constraint corresponding to the energy consumption cost are dynamically weighted and adjusted. Based on the state value estimation, the delivery reliability constraint after dynamic weight adjustment, the delay constraint after dynamic weight adjustment, and the energy consumption constraint after dynamic weight adjustment, the parameters of the distributed routing decision model to be trained are updated through adaptive constraint weights or Lagrange multiplier update methods to obtain the trained distributed routing decision model.

[0013] To achieve the above objectives, another aspect of the present invention proposes a constrained sensing graph-enhanced routing decision-making device for underwater acoustic sensor networks, the device comprising: The network status acquisition module is used to obtain local observation information of the current forwarding node; The feasible link determination module is used to perform multi-dimensional feasibility determination on candidate neighbor nodes based on the local observation information and generate a set of feasible next hops; The action masking module is used to input the local observation information and the feasible next-hop set into the distributed routing decision model to be trained, and output pre-trained routing actions. The training phase route execution and state update module is used to execute the pre-trained route actions, obtain the execution results, and update the local observation information based on the execution results; The training period graph value assessment module is used to construct a dynamic feasible graph using the updated local observation information, perform graph augmentation value assessment on the dynamic feasible graph, and obtain a state value estimate. The constraint update module is used to perform constraint-aware training on the distributed routing decision model to be trained using the state value estimation and the execution result, so as to obtain the trained distributed routing decision model. The deployment module is used to deploy the trained distributed routing decision model to underwater nodes, input the local observation information to be decided and the set of feasible next hops to be decided into the trained distributed routing decision model, and output the target routing action.

[0014] To achieve the above objectives, another aspect of the present invention provides an electronic device, the electronic device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method described above.

[0015] To achieve the above objectives, another aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described above.

[0016] To achieve the above objectives, another aspect of the present invention provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions to cause the computer device to perform the aforementioned method.

[0017] The embodiments of the present invention include at least the following beneficial effects: The present invention provides a constraint-aware graph-enhanced routing decision method and related equipment for underwater acoustic sensor networks. This scheme obtains local observation information of the current forwarding node to provide a data foundation for subsequent steps; based on the local observation information, it performs multi-dimensional feasibility judgment on candidate neighbor nodes to generate a feasible next-hop set; it inputs the local observation information and the feasible next-hop set into a distributed routing decision model to be trained, outputs pre-trained routing actions, restricts the routing decision to the feasible candidate range, and avoids the model from making invalid selections of infeasible nodes; it executes the pre-trained routing actions to obtain execution results, and updates the local observation information based on the execution results, providing real-time and accurate observation data for subsequent training and decision-making; through the updated... Based on local observation information, a dynamic feasible graph is constructed. Graph augmentation value assessment is performed on the dynamic feasible graph to obtain a state value estimate. Global topology information is introduced during the training phase to improve the accuracy of the value estimate and compensate for the estimation bias caused by using only local observations for decision-making. Using the state value estimate and the execution results, the distributed routing decision model to be trained is constrained and trained to obtain the trained distributed routing decision model. By coordinating various constraints into the optimization objective and dynamically adjusting the constraint weights, various performance trade-offs are achieved. The trained distributed routing decision model is deployed to an underwater node. The local observation information to be decided and the set of feasible next hops to be decided are input into the trained distributed routing decision model, and the target routing action is output, thereby improving routing stability and energy utilization efficiency. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a constrained sensing graph-enhanced routing decision method for underwater acoustic sensor networks provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a hop-by-hop routing scenario for an underwater acoustic sensor network provided in an embodiment of the present invention; Figure 3 This is a flowchart of the feasible next-hop action set generation and action masking provided in the embodiments of the present invention; Figure 4 This is a flowchart of the constraint-aware strategy optimization provided in an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the relationship between the training phase and the execution phase modules provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the overall process of constrained perception graph-enhanced routing decision-making for underwater acoustic sensor networks provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the module composition of the constraint perception graph enhanced routing decision device for underwater acoustic sensor networks provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this invention; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this invention as detailed in the appended claims.

[0021] It should be noted that although functional modules are divided in the system diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first / S100" and "second / S200" in the specification, claims, and the foregoing drawings may be used herein to describe various concepts, but unless specifically stated otherwise, these concepts are not limited by these terms. These terms are used only to distinguish one concept from another. For example, first information may also be referred to as second information without departing from the scope of the embodiments of the invention, and similarly, second information may also be referred to as first information. Depending on the context, the words "if" or "when" as used herein may be interpreted as "when," "in response to a determination," or "in the event of a determination."

[0022] The terms “at least one,” “multiple,” “each,” “any,” etc., used in this invention, “at least one” includes one, two, or more than two; “multiple” includes two or more than two; “each” refers to each of the corresponding multiple; and “any” refers to any one of the multiple.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0024] Traditional underwater routing protocols often select the next hop based on geographical location, depth, pressure, progress of the sink node, or local neighbor information. These methods are simple to implement, have low control overhead, and are highly distributed, making them suitable for resource-constrained underwater nodes. However, traditional rule-based methods typically rely on manually set local ranking rules or thresholds, which present the following problems in complex dynamic environments: First, obvious local greedy decision-making: Traditional methods often select based only on the distance, depth progress, or local link quality from the current node to candidate nodes, making it difficult to comprehensively consider subsequent route reachability, global topology, and future energy consumption. Second, insufficient characterization of system-level constraints: Packet delivery success rate, end-to-end latency, and node energy consumption are often not uniformly incorporated into the same optimization framework, leading to situations where some methods may improve local forwarding progress but cause a decrease in overall delivery rate or an increase in energy consumption. Third, prominent issues of gaps and energy imbalances: Under conditions of sparse topology, uneven node energy, or fluctuating channel quality, routing is prone to problems such as no feasible next hop, repeated detours, or overload of a few relay nodes. Fourth, parameter adjustment relies on experience: communication radius, forwarding priority, suppression time, threshold parameters, etc., usually need to be manually adjusted according to the scenario, which is difficult to adapt to deployment needs of different scales, water depths, and link environments.

[0025] In recent years, reinforcement learning and deep reinforcement learning have been applied to underwater network routing, enabling nodes to learn routing strategies through interaction with the environment. Compared to traditional rule-based methods, learning-based methods have stronger adaptability, adjusting next-hop selection based on node energy, link state, queue state, and historical forwarding results. However, existing learning-based routing methods still have the following shortcomings: First, many methods combine delivery success, latency, and energy consumption into a single reward value for optimization, lacking explicit modeling of different types of constraints such as delivery failure, latency exceeding limits, and energy consumption exceeding budget. The resulting strategy may compromise other key metrics while improving one. Second, the set of available next hops in an underwater network dynamically changes with communication distance, remaining node energy, link latency, link quality, queue load, and access history. If the policy model directly outputs action probabilities to all nodes, it is easy to assign selection probabilities to unreachable, low-energy, or latency-inadequate nodes, leading to training instability or invalid actions during execution. Third, real-world underwater nodes typically cannot obtain the global network topology and the states of all nodes, requiring a lightweight, localized, and distributed routing decision-making mechanism during the execution phase. However, if only local information is used during the training phase, value estimation may be inaccurate, affecting policy convergence speed and generalization ability. Fourth, when there are no candidate next hops that satisfy the constraints for the current node, some learning strategies lack explicit safety backoff handling and may continue to output invalid next hops, resulting in additional forwarding attempts, waiting, retransmissions, and energy waste.

[0026] In view of this, this invention provides a constraint-aware graph-enhanced routing decision-making method and related equipment for underwater acoustic sensor networks. This scheme constructs a set of feasible next-hop actions for the current forwarding node, combining action masking mechanisms, safe backoff actions, a training-period graph enhancement value assessment mechanism, and a constraint-aware strategy optimization mechanism. This allows underwater nodes to make hop-by-hop routing decisions based solely on local observations during the execution phase, while simultaneously improving the quality of strategy learning during the training phase using network topology and global state information. For example, this scheme uses the process of forwarding a data packet hop-by-hop from the source node to the sink node as a routing process. In each hop, the underwater node currently holding the data packet acts as the current forwarding node, generating a set of feasible next-hops based on its own and its neighboring nodes' local states, and calling a distributed routing decision-making module to select the next hop. During the training phase, a dynamic feasible graph of the network is constructed, and the value of the routing state is estimated using the training-period graph enhancement value assessment module. Then, constraint-aware training of the routing strategy is performed, incorporating energy consumption, latency, and delivery failure costs. During the execution phase, ordinary underwater nodes are not required to acquire the global topology, nor are they required to run the training period graph augmentation value assessment module. Only the distributed routing decision module and the feasibility judgment module are deployed.

[0027] This solution can reduce unnecessary forwarding, detours, retransmissions and energy consumption while ensuring the reliability of data packet delivery, thereby improving the routing stability, energy efficiency and practical deployment feasibility of underwater acoustic sensor networks in dynamic environments.

[0028] The constraint-aware graph-enhanced routing decision method for underwater acoustic sensor networks provided in this invention relates to the field of wireless sensor network technology. This method can be applied to terminals, servers, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or vehicle-mounted terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the constraint-aware graph-enhanced routing decision method for underwater acoustic sensor networks, but is not limited to the above forms.

[0029] Figure 1 This is an optional flowchart of a constraint-aware graph-enhanced routing decision method for underwater acoustic sensor networks provided in an embodiment of the present invention. Figure 1 The method may include, but is not limited to, steps S100 to S700: Step S100: Obtain local observation information of the current forwarding node; Step S200: Based on local observation information, perform multi-dimensional feasibility judgment on candidate neighbor nodes and generate a feasible next-hop set; Step S300: Input the local observation information and the set of feasible next hops into the distributed routing decision model to be trained, and output the pre-trained routing actions; Step S400: Execute the pre-trained routing action, obtain the execution result, and update the local observation information based on the execution result; Step S500: Construct a dynamic feasible graph using the updated local observation information, perform graph augmentation value assessment on the dynamic feasible graph, and obtain a state value estimate; Step S600: Using state value estimation and execution results, perform constraint-aware training on the distributed routing decision model to be trained to obtain the trained distributed routing decision model. Step S700: Deploy the trained distributed routing decision model to the underwater node, input the local observation information to be decided and the set of feasible next hops to be decided into the trained distributed routing decision model, and output the target routing action.

[0030] Before step S100 in some embodiments, such as Figure 2 As shown, an underwater acoustic sensor network was constructed, comprising multiple underwater sensor nodes (i.e., underwater nodes) and at least one sink node. Each underwater sensor node has a node identifier, spatial location, remaining energy, queue load, communication capability, neighbor node status table, historical access records, and local timing status. The sink node is used to receive data packets from the underwater sensor nodes and can be configured as a surface buoy, fixed base station, underwater gateway, or mobile sink node.

[0031] In a routing process, the source node generates a data packet to be sent. The data packet is held by the source node or the current relay node, and the current packet-holding node selects the next-hop node according to the routing decision method of the present invention. If the data packet arrives at the sink node, the routing is successful; if a link forwarding failure, insufficient energy, end-to-end delay exceeding the limit, no feasible next hop, reaching the maximum number of hops, or an abnormal loop occurs, the current routing process ends, and the reason for termination is recorded.

[0032] In step S100 of some embodiments, local observation information of the current forwarding node is obtained, that is, the current forwarding node obtains its own state information and the local state information of candidate neighbor nodes. Optionally, the local observation information includes at least one or more of the following: the remaining energy, depth, or location information of the current forwarding node; the relative distance or forwarding progress information between the current forwarding node and the sink node; the remaining energy, depth, location, or load status of candidate neighbor nodes; the communication reachability status between the current forwarding node and candidate neighbor nodes; the estimated single-hop delay, link quality, or successful forwarding probability of the candidate link; the cumulative delay, cumulative energy consumption, cumulative hop count, or access history of the current forwarding routing process; whether the candidate node has been visited, whether there is a loop risk, or whether it has the potential to continue forwarding to the sink node; the proportion of currently feasible candidate actions, the number of neighbor nodes, or the local congestion status.

[0033] The aforementioned local observation information can be obtained through underwater node local sensing, neighbor status broadcasting, link detection, underwater acoustic modem feedback, historical forwarding statistics, network management information, or simulation / digital twin systems. To reduce communication overhead, neighbor status information can be maintained using periodic broadcasting, on-demand broadcasting, event-triggered broadcasting, or compressed encoding methods.

[0034] In step S200 of some embodiments, the current forwarding node performs a feasibility assessment on each candidate neighbor node and generates a set of feasible next hops corresponding to the current routing process. Optionally, as shown... Figure 3 As shown, a candidate neighbor node is determined as a feasible next hop when it meets at least some of the following conditions: the candidate neighbor node is not the current node itself; the current forwarding node and the candidate neighbor node meet the communication distance, signal-to-noise ratio, link quality, or other communication reachability requirements; after the current forwarding node completes one or more allowed transmission attempts, its remaining energy is not less than a preset transmission energy threshold; after the candidate neighbor node becomes a receiving node, its remaining energy is not less than a preset receiving energy threshold; the estimated single-hop delay of the candidate link does not exceed a preset single-hop delay upper limit; the candidate neighbor node will not cause significant route backtracking, repeated access, local loops, or invalid detours; the candidate neighbor node has positive forwarding progress relative to the sink node, or meets preset detour conditions in a hole detour scenario; the queue load or congestion status of the candidate neighbor node is lower than a preset threshold, or meets an acceptable load level. Based on the above judgments, a feasibility flag is set for each candidate action (i.e., sending a data packet from the current forwarding node to a candidate neighbor node). Feasible candidate actions are allowed to be selected by the policy, while infeasible candidate actions are blocked during the action selection phase. The specific threshold for feasibility assessment can be set based on the network deployment area, communication radius, performance of underwater acoustic modems, energy budget, service latency requirements, and network scale.

[0035] In some embodiments, step S200 may include, but is not limited to, steps S210 to S270: Step S210: Based on local observation information, extract the first remaining energy of the current forwarding node, the second remaining energy of the candidate neighbor nodes, the estimated single-hop delay of the candidate links, and the access history in the cumulative state of the current routing process. Step S220: Determine whether the candidate neighbor node is within the communication coverage area of ​​the current forwarding node to obtain the first determination result; Step S230: Determine whether the first remaining energy is greater than or equal to the preset transmission energy threshold, and obtain the second determination result; Step S240: Determine whether the second remaining energy is greater than or equal to the preset received energy threshold, and obtain the third determination result; Step S250: Determine whether the estimated single-hop delay is less than the preset single-hop delay upper limit, and obtain the fourth determination result; Step S260: Determine whether the candidate neighbor node exists in the access history and obtain the fifth judgment result; Step S270: Generate a feasible next-hop set based on the first judgment result, the second judgment result, the third judgment result, the fourth judgment result, and the fifth judgment result.

[0036] In step S210 of some embodiments, core parameters for multi-dimensional judgment are parsed and extracted from the data pool of local observation information. These parameters may include, but are not limited to, extracting the remaining energy of the current forwarding node itself as the first remaining energy, extracting the estimated remaining energy of the candidate neighbor node after receiving data as the second remaining energy, extracting the estimated time of single-hop data transmission of the candidate link obtained by the current detection as the estimated single-hop delay, and extracting the list of node IDs that the current data packet has visited in this routing process as the access history. These parameters are the basic data source for subsequent execution of specific constraint judgments, ensuring the objectivity and real-time nature of the judgment basis.

[0037] In step S220 of some embodiments, it is detected whether the candidate neighbor node is within the effective underwater acoustic communication radius of the current forwarding node, or by measuring the link signal-to-noise ratio, bit error rate and handshake response, it is determined whether there is an available physical link connection between the two. If the candidate node is not within the communication coverage area (the first judgment result is no), it is directly marked as an infeasible candidate node. This judgment serves as the first threshold for feasibility filtering, effectively avoiding transmission failure and energy waste caused by physical layer unreachability.

[0038] In step S230 of some embodiments, the first remaining energy of the current forwarding node is compared with a preset transmission energy threshold, which covers the startup energy consumption and transmission energy consumption required to drive the underwater acoustic transducer to transmit data packets; if the first remaining energy is lower than the transmission energy threshold (the second judgment result is no), it indicates that the current node cannot complete the transmission operation. This judgment prevents the node from failing due to energy depletion caused by forced transmission, thus ensuring the node's own survival time.

[0039] In step S240 of some embodiments, the second remaining energy of the candidate neighbor node is compared with a preset receiving energy threshold. The receiving energy threshold ensures that the candidate node has the minimum required energy to receive data packets, perform data verification, and participate in network operation. If the second remaining energy is lower than the receiving energy threshold (the third judgment result is no), it is marked as an infeasible candidate node. This avoids data packet loss or the formation of new routing breakpoints due to selecting a node with insufficient energy as the next hop, and helps to balance the energy consumption of the entire network.

[0040] In step S250 of some embodiments, the estimated single-hop delay of the candidate link is compared with a preset single-hop delay upper limit, which is derived from the end-to-end delay budget of the service combined with the current number of hops; if the estimated single-hop delay exceeds the upper limit (the fourth judgment result is no), it is determined that the link cannot meet the delay constraint and is excluded. This judgment is a key means to ensure that the data packet arrives at the aggregation node within the effective life cycle, which meets the low latency requirements of underwater services.

[0041] In step S260 of some embodiments, the access history of the current routing process is retrieved, and it is determined whether the identifier of the candidate neighbor node already exists in the list. If the candidate neighbor node already exists in the access history (the fifth determination result is yes), it is determined that selecting the node will cause a routing loop or invalid backoff, and the candidate neighbor node is marked as an infeasible candidate node. This determination ensures the directionality and loop-free nature of the route through a simple historical comparison mechanism.

[0042] In step S270 of some embodiments, a logical AND operation is performed on the Boolean judgment results of the five dimensions. Only when all judgment results are "yes" (i.e., all constraints are met) is the candidate neighbor node included as a feasible candidate node in the feasible next-hop set. The final generated set contains all feasible candidate nodes that meet the constraints of communication, energy, latency and acyclicity at the current time. This provides accurate input basis for the action shielding mechanism of the subsequent distributed routing decision model, ensuring that the model makes optimal decisions within a safe range.

[0043] In step S300 of some embodiments, such as Figure 3 As shown, to avoid selecting infeasible actions in routing policy selection, this invention introduces an action masking mechanism in the action output stage of the distributed routing decision model. This mechanism generates an action mask based on the set of feasible next hops, preventing infeasible candidate nodes from participating in the selection process during action probability normalization, action scoring ranking, or action sampling. Thus, the effective selection range for routing decisions is limited to the set of candidate neighbor nodes that currently meet the constraints. When the current forwarding node does not have a feasible next hop that meets the constraints, the action masking mechanism activates a safe fallback action. Optionally, the safe fallback action can be set to one or more of the following, depending on the application scenario: terminating the forwarding of the current data packet and recording the reason for the routing failure; reporting the failure status to the network management node or upper-layer protocol; triggering a hole recovery or abnormal detour process; requesting the source node or aggregation node to reschedule; caching the data packet and waiting for the next routing cycle; or switching to a preset fallback routing rule.

[0044] In some optional embodiments, a safety fallback action is used to terminate the current routing process to avoid continuous waiting, repeated attempts, or invalid actions when no feasible links are available. This setting can reduce unnecessary energy consumption and latency, and allow failure events during training to be explicitly fed back to the policy optimization mechanism.

[0045] In some optional embodiments, the distributed routing decision model is a shared parameter model, deployed in underwater nodes, edge control devices, network controllers, or simulation control terminals, and invoked only by the node currently holding the data packet in each routing process. The model's input includes local observation information of the current forwarding node and feasibility indicators of candidate actions, and its output is a routing action to send the data packet to the next-hop node or a safe backoff action. Optionally, the distributed routing decision model can be implemented using neural networks, policy tables, function approximators, lightweight classifiers, ranking-based policy models, or combinations thereof. When implemented using neural networks, the model may include local state encoding units, candidate action feature encoding units, action scoring units, and action masking output units. In subsequent execution phases, different underwater nodes can use the same shared parameter model. The current forwarding node only needs to construct inputs based on locally available information to independently select the next hop, thereby avoiding the underwater acoustic communication overhead caused by global state broadcasting.

[0046] In some embodiments, step S300 may include, but is not limited to, steps S310 to S350: Step S310: When the feasible next hop set is not empty, generate an action mask based on the feasible next hop set; Step S320: Input the local observation information into the feature encoding unit of the distributed routing decision model to be trained, and output the candidate action feature vector; each component in the candidate action feature vector corresponds to each candidate neighbor node; Step S330: Based on the action mask, perform action masking on infeasible candidate nodes in the candidate action feature vector to obtain the action score vector; Step S340: Normalize the action score vector to obtain the selection probability distribution of each feasible candidate node in the feasible next jump set; Step S350: Select the feasible candidate node with the highest probability in the selection probability distribution as the next hop node to obtain the pre-trained routing action.

[0047] In step S310 of some embodiments, when the feasible next-hop set is not empty, a binary action mask vector (MaskVector) with the same dimension as the candidate action space is generated according to the index position of the element in the feasible next-hop set. In this action mask vector, the position corresponding to the feasible candidate node in the feasible next-hop set is set to 1 (indicating that selection is allowed), while the position corresponding to the infeasible candidate node that has been eliminated by the feasibility judgment is set to 0 (indicating that selection is prohibited).

[0048] In step S320 of some embodiments, local observation information, including the remaining energy of the current forwarding node, queue length, distance to the sink node, and state information of candidate neighbor nodes, is input to the feature encoding unit (usually composed of multiple fully connected neural networks or convolutional layers) of the distributed routing decision model to be trained. After nonlinear transformation and feature fusion, this unit outputs a candidate action feature vector. Each dimension of this candidate action feature vector corresponds one-to-one with a candidate neighbor node, and its value implies the potential value of the candidate neighbor node as the next hop (such as link quality, direction of advance, etc.), but the feasibility constraints of the action have not yet been considered.

[0049] In step S330 of some embodiments, the candidate action feature vector is multiplied element-wise with the action mask. After the element-wise multiplication operation, the feature components corresponding to infeasible candidate nodes with a mask value of 0 are forced to zero or assigned negative infinity, thereby numerically completely masking the output possibility of these illegal actions; while the feature components corresponding to feasible candidate nodes with a mask value of 1 are retained, forming the final action score vector. This action score vector intuitively reflects the forwarding priority of each feasible candidate node under the premise of complying with all constraints.

[0050] In step S340 of some embodiments, to facilitate random sampling or deterministic selection, the action score vector is normalized to transform it into a probability distribution. Optionally, a Softmax function is applied to the action score vector for normalization. This Softmax function maps each score value in the action score vector to a continuous interval between 0 and 1, ensuring that the sum of the values ​​corresponding to all feasible candidate nodes is 1. The resulting output is the selection probability distribution of each feasible candidate node, which clearly characterizes the model's confidence in each candidate node as the optimal next hop in the current network state.

[0051] In step S350 of some embodiments, the probability distribution is analyzed. Optionally, a greedy strategy or direct selection of the feasible candidate node with the largest value in the probability quality function is adopted as the next-hop node. This next-hop node is the pre-trained routing action selected at the current decision moment. This probability-based selection mechanism not only ensures that the network environment can be fully explored in the early stage of training, but also ensures that the learned knowledge can be used to select the optimal path in the later stage of training, thereby achieving efficient packet forwarding.

[0052] In some embodiments, the constraint-aware graph-enhanced routing decision method for underwater acoustic sensor networks further includes: When the feasible next-hop set is empty, a safe backoff action is activated; in response to the safe backoff action, the current routing process is terminated. For example, when, after multi-dimensional feasibility assessment, the feasible next-hop set is empty (i.e., there are no neighboring nodes that meet the energy, congestion, and link quality requirements), the system automatically activates a preset safe backoff action. In response to this safe backoff action, the current forwarding node will terminate the current routing process, effectively preventing nodes from blindly forwarding when no available next-hop node exists, significantly improving network robustness.

[0053] In step S400 of some embodiments, the pre-trained routing action is executed. If the pre-trained routing action is next-hop forwarding, data packets are sent through the underwater acoustic link and the sending time is recorded, while waiting for an acknowledgment frame from the receiving node (i.e., the next-hop node). If the pre-trained routing action is safe rollback, the current routing process is terminated and the reason for failure is recorded. After execution, the measured execution results are collected, including at least a delivery failure flag (0 for success, 1 for failure), the actual cumulative delay of the current routing process, and the actual energy consumption increment of this forwarding. Subsequently, the local observation information is updated based on the execution results, including deducting the corresponding energy consumption of the current forwarding node and the receiving node, refreshing the link quality estimate, updating the queue load status, and appending the current forwarding node to the access history. The updated local observation information will serve as the input for subsequent dynamic feasible graph construction and the next round of decision-making, ensuring that the network state matches the underwater dynamic environment in real time.

[0054] In step S500 of some embodiments, the underwater acoustic sensor network is represented as a dynamic feasible graph. Nodes in the graph correspond to underwater sensing nodes and sink nodes, and directed edges correspond to feasible forwarding links that currently satisfy communication constraints, energy constraints, and latency constraints. Since node remaining energy, queue load, cumulative latency, and link quality change during the forwarding process, the dynamic feasible graph is also dynamically updated with each routing step. A graph augmentation value assessment module (i.e., a training-phase graph value assessment module) is constructed to assess the value of the current routing state based on the global network state and the dynamic feasible graph during the training phase. The graph augmentation value assessment module can receive one or more of the following training information: state characteristics of all nodes; adjacency relationships in the current dynamic feasible graph; current packet-holding node indication information; sink node indication information; source node or historical access indication information; cumulative latency, cumulative energy consumption, hop count, feasible action ratio, and current node energy state in the current routing process.

[0055] In some optional embodiments, the graph augmentation value assessment module can be implemented using graph neural networks, graph convolutional networks, graph attention networks, message passing networks, neighbor aggregation networks, or other models capable of processing graph structure information. Its role is to provide more accurate value estimates or auxiliary training signals during the training phase. During the execution phase, ordinary underwater nodes do not need to run this module, nor do they need to acquire the global graph structure, thus maintaining a lightweight deployment.

[0056] In some embodiments, step S500 may include, but is not limited to, steps S510 to S530: Step S510: Extract the state features of underwater nodes and sink nodes from the updated local observation information, and extract the feasible forwarding link relationships between underwater nodes and between underwater nodes and sink nodes. Step S520: Based on the underwater nodes and the aggregation nodes, construct a graph node set; based on the feasible forwarding link relationships, construct a directed edge set; and based on the graph node set and the directed edge set, obtain a dynamic feasible graph. Step S530: Based on the state characteristics, feasible forwarding link relationships, the first indication information of the current forwarding node, and the second indication information of the aggregation node, perform graph augmentation value assessment on the dynamic feasible graph to obtain a state value estimate.

[0057] In step S510 of some embodiments, the state characteristics of all underwater sensing nodes and sink nodes are first extracted, including the node's remaining energy, queue load, distance to the sink node, communication capability, etc. These characteristics directly reflect the node's operating status and forwarding potential. Secondly, the feasible forwarding link relationship between nodes is extracted, that is, only links that simultaneously meet the three constraints of communication reachability, sufficient energy margin, and single-hop delay meet the requirements are retained, and all invalid links that do not meet the constraints are excluded to ensure the rationality of the subsequent graph structure.

[0058] In step S520 of some embodiments, all underwater sensing nodes and aggregation nodes are aggregated into a graph node set, with each node corresponding to a set of state features. Then, a set of directed edges is constructed based on feasible forwarding link relationships. The starting point of each directed edge is the node currently capable of forwarding, and the ending point is its corresponding feasible next-hop node. The edge attributes include features such as link latency and link quality. Finally, the graph node set and the directed edge set together constitute a dynamic feasible graph. This graph is updated in real time as the routing process progresses. Once node energy decays, link quality fluctuates, or queue load changes, the corresponding node features or edge relationships will be adjusted synchronously to accurately match the time-varying characteristics of the underwater channel.

[0059] In step S530 of some embodiments, the dynamic feasible graph, node state features and feasible forwarding link relationships, as well as the current forwarding node indication information (typically identified by one-hot encoding of the current packet-holding node) and the aggregation node indication information (identified by one-hot encoding of the data delivery target) are jointly input into the graph augmentation value assessment module. This module aggregates neighbor node features through mechanisms such as graph convolution, graph attention, or message passing, fuses global topology information and local state features, and outputs a state value estimate of the current routing state. This state value estimate measures the long-term benefits of the current routing state from a global perspective, providing a reliable benchmark reference for subsequent constraint-aware training and avoiding value estimation bias caused by local observations.

[0060] like Figure 4 , Figure 5 As shown, during the training phase of the distributed routing decision model, training feedback signals are constructed based on factors such as successful packet delivery, forwarding progress to the aggregation node, link reliability, energy consumption, latency accumulation, repeated access, and termination reasons. Simultaneously, energy consumption cost, latency exceeding the limit cost, and delivery failure cost are introduced as constraint cost signals into the policy optimization process. In some embodiments, adaptive constraint weights or Lagrange multiplier updates are used to dynamically adjust their impact on the policy optimization objective based on the energy cost, latency cost, and delivery failure cost observed in the training batch. When a certain type of cost exceeds the preset budget, the weight of the corresponding constraint term is increased; when the cost is lower than the preset budget, the weight of the corresponding constraint term is decreased or maintained. Through this method, the routing policy can improve delivery reliability while suppressing end-to-end latency and energy consumption.

[0061] Optionally, policy optimization can be achieved using proximal policy optimization, constrained policy optimization, actor-critic methods, policy gradient methods, value function approximation methods, or other reinforcement learning optimization methods. After training, the trained distributed routing decision model is retained for the execution phase, while the graph augmentation value evaluation module is only retained in the training environment as an auxiliary module during the training period.

[0062] In some embodiments, step S600 may include, but is not limited to, steps S610 to S630: Step S610: Obtain the delivery failure cost, latency cost, and energy consumption cost from the execution results; Step S620: Based on the first comparison result of delivery failure cost and the first preset budget, the second comparison result of delay cost and the second preset budget, and the third comparison result of energy consumption cost and the third preset budget, dynamically adjust the weights of the delivery reliability constraint corresponding to delivery failure cost, the delay constraint corresponding to delay cost, and the energy consumption constraint corresponding to energy consumption cost. Step S630: Based on the state value estimation, the delivery reliability constraint after dynamic weight adjustment, the delay constraint after dynamic weight adjustment, and the energy consumption constraint after dynamic weight adjustment, the parameters of the distributed routing decision model to be trained are updated by adaptive constraint weight or Lagrange multiplier update method to obtain the trained distributed routing decision model.

[0063] In step S610 of some embodiments, three types of quantified costs are extracted from the execution result, including: delivery failure cost, which is directly taken from the delivery failure flag in the execution result, with a value of 0 (delivery successful) or 1 (delivery failed); latency cost, which is taken from the actual cumulative latency of the current routing process in the execution result, reflecting the real-time latency overhead of packet forwarding; and energy consumption cost, which is taken from the actual energy consumption increment in the execution result, reflecting the energy consumption of this routing process, and providing an objective basis for the dynamic adjustment of subsequent constraint weights.

[0064] In step S620 of some embodiments, the three types of costs are compared with their corresponding preset budgets, including: comparing the delivery failure cost with a first preset budget (usually set to 0, i.e., delivery failure is not allowed); if the delivery failure cost is 1, the weight of the delivery reliability constraint is increased; comparing the latency cost with a second preset budget (maximum allowable end-to-end latency); if the latency cost exceeds the budget, the weight of the latency constraint is increased, otherwise the weight is decreased or maintained; and comparing the energy consumption cost with a third preset budget (maximum allowable energy consumption per packet); if the energy consumption cost exceeds the budget, the weight of the energy consumption constraint is increased, otherwise the weight is decreased or maintained. This adjustment mechanism is entirely based on actual measured costs and is adaptively executed without manual intervention, making it adaptable to network deployment scenarios of different scales and business needs.

[0065] In step S630 of some embodiments, the state value estimate is used as the baseline value, and weighted terms for delivery reliability constraints, latency constraints, and energy consumption constraints, adjusted by dynamic weights, are superimposed to construct the final policy optimization objective. Subsequently, reinforcement learning optimization methods such as near-end policy optimization and actor-commentator are used to calculate the policy gradient, and all network parameters of the distributed routing decision model are updated through backpropagation. After multiple rounds of iterative training, the model can learn the optimal routing policy under different network states, effectively reducing end-to-end latency and node energy consumption while ensuring delivery reliability. Finally, a trained distributed routing decision model is obtained for subsequent underwater node deployment.

[0066] In step S700 of some embodiments, the trained distributed routing decision model is first lightweighted and adapted before deployment. Optionally, the model size is compressed and the computational complexity is reduced through processing methods such as low bit quantization, model pruning, and operator fusion, so that it can adapt to the limited computing power, storage, and energy consumption constraints of the embedded processors of underwater nodes. All underwater nodes load the same lightweight model with shared parameters, without the need for separate customization for node location and role, adapting to the network deployment needs of different scales. When it is necessary to forward data packets, each underwater node independently executes the local decision-making process, including: firstly, collecting local observation information to be decided in real time, including the node's remaining energy, depth, relative distance to the sink node, status and load of candidate neighbor nodes, latency and quality of candidate links, cumulative cost and access history of the current routing process, etc., which is completed entirely based on local perception data, without the need to interact with other nodes for global topology information; then, performing multi-dimensional feasibility judgment based on the local observation information to be decided, generating a set of feasible next hops to be decided and the corresponding binary action mask, ensuring that only candidate nodes that meet the constraints of communication, energy, and latency are included in the decision scope. The local observation information and action mask to be decided are then input into the trained distributed routing decision model. The model reuses the feature encoding and action selection logic learned during the training phase: high-dimensional features of local observations are extracted through the feature encoding unit; the scores of infeasible candidate nodes are forced to zero based on the action mask; the selection probability distribution of each candidate node is output within the feasible next-hop set; and finally, the feasible candidate node with the highest probability is selected as the next-hop output. If the feasible next-hop set is empty, a safe backoff action is directly output, resulting in the final target routing action. This step retains the comprehensive optimization capabilities of reliability, latency, and energy consumption learned during the training phase while completely avoiding the underwater acoustic communication overhead caused by global information interaction. It adapts to the characteristics of narrow bandwidth and high latency in underwater channels and can dynamically respond to scenarios such as node energy attenuation, link quality fluctuations, and topology changes, ensuring the real-time performance and effectiveness of routing decisions.

[0067] In some implementations, such as Figure 5 , Figure 6As shown, during the actual routing execution phase, after the source node generates a data packet, the current packet-holding node performs the following process: It acquires local observation information such as its own state, neighbor node states, and candidate link states; it generates a feasible next-hop set based on communication, energy, latency, and route validity constraints; it generates an action mask identifier (i.e., an action mask) based on the feasible next-hop set; it inputs the local observation information and the action mask identifier into the trained distributed routing decision module; if a feasible next hop exists, it selects one of the nodes as the next hop and performs underwater acoustic link forwarding; if no feasible next hop exists, it performs a safe fallback action; it updates the sending node energy, receiving node energy, cumulative latency, cumulative energy consumption, queue load, and access history based on the forwarding results; if the data packet reaches the sink node, the routing ends successfully; if forwarding failure, timeout, insufficient energy, reaching the maximum hop count, or an abnormal loop occurs, the routing ends in failure; otherwise, the new current packet-holding node continues the above process.

[0068] In some optional embodiments, the present invention is not limited to a specific reinforcement learning algorithm, a specific graph neural network structure, or a specific underwater acoustic link model. Without changing the core idea of ​​the invention, the following substitutions can be made: the distributed routing decision module can be implemented by a probabilistic policy model, a deterministic policy model, a ranking model, or a rule-learning hybrid model; the training-period graph reinforcement value evaluation module can be implemented by graph convolution, graph attention, message passing, neighbor average aggregation, or other graph coding methods; feasibility judgment can be based on a combination of distance, signal-to-noise ratio, bit error rate, channel estimation, remaining energy, node load, single-hop latency, or service priority; the safety fallback action can be selected according to the application scenario, including termination, caching, rescheduling, hole recovery, or switching to a backup route; the constraint optimization mechanism can be implemented using adaptive weights, Lagrange updates, penalty functions, constraint policy optimization, or multi-objective optimization methods; the aggregation node can be a fixed node, a mobile node, a surface buoy, an autonomous underwater vehicle, or a shore-based gateway; the present invention can be applied to single-data packet routing, multi-service priority routing, clustered network routing, or routing scenarios that are co-optimized with media access control protocols.

[0069] like Figure 7 As shown, this embodiment of the invention also provides a constraint-aware graph-enhanced routing decision-making device for underwater acoustic sensor networks, which can implement the above-mentioned constraint-aware graph-enhanced routing decision-making method for underwater acoustic sensor networks. The device includes: The network status acquisition module is used to obtain local observation information of the current forwarding node; The feasible link judgment module is used to make multi-dimensional feasibility judgments on candidate neighbor nodes based on local observation information and generate a set of feasible next hops; The action masking module is used to input local observation information and the set of feasible next hops into the distributed routing decision model to be trained, and output pre-trained routing actions. The training phase routing execution and state update module is used to execute pre-trained routing actions, obtain execution results, and update local observation information based on the execution results; The training period graph value assessment module is used to construct a dynamic feasible graph using updated local observation information, perform graph augmentation value assessment on the dynamic feasible graph, and obtain a state value estimate. The constraint update module is used to perform constraint-aware training on the distributed routing decision model to be trained using state value estimation and execution results, so as to obtain the trained distributed routing decision model. The deployment module is used to deploy the trained distributed routing decision model to underwater nodes. It inputs the local observation information to be decided and the set of feasible next hops to be decided into the trained distributed routing decision model and outputs the target routing action.

[0070] A constraint-aware graph-enhanced routing decision-making device for underwater acoustic sensor networks may further include: The routing execution module in the execution phase is used to control the forwarding of data packets hop-by-hop between underwater nodes and output statuses such as routing success, link failure, no feasible next hop, timeout, insufficient energy, or abnormal loop based on the forwarding results.

[0071] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0072] This invention also provides an electronic device, which includes a processor and a memory. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including a tablet computer, an in-vehicle computer, or similar device.

[0073] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0074] refer to Figure 8 , Figure 8 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 801 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention. The memory 802 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 802 and is called and executed by the processor 801. The 803 input / output interface is used to implement information input and output. The communication interface 804 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 805 transmits information between various components of the device (e.g., processor 801, memory 802, input / output interface 803, and communication interface 804); The processor 801, memory 802, input / output interface 803, and communication interface 804 are connected to each other within the device via bus 805.

[0075] In practical deployments, electronic devices can be embedded processors in underwater sensing nodes, edge gateways, surface buoy controllers, shore-based servers, simulation training servers, or network management platforms. The models required for the execution phase can be compressed, quantized, or lightweighted before being deployed on resource-constrained devices.

[0076] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0077] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0078] This invention also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions to cause the computer device to perform the aforementioned method.

[0079] In summary, the constraint-aware graph-enhanced routing decision method and related equipment for underwater acoustic sensor networks according to embodiments of the present invention have the following advantages: 1. Reduce the risk of unavailable action selection: By using a feasible next-hop set and action masking mechanism, the strategy selects only from candidate nodes that meet the constraints of communication, energy and time delay, avoiding the misselection of unreachable nodes, nodes with insufficient energy or timeout nodes.

[0080] 2. Improve routing execution security: When no feasible next hop exists, the safe fallback action can terminate in time or enter the abnormal handling process, reducing invalid forwarding, repeated attempts and extra waiting.

[0081] 3. Balancing training effectiveness and lightweight deployment: During the training phase, dynamic feasible graphs and global node states are used to improve the quality of value estimation. During the execution phase, only a distributed routing decision module is deployed to reduce the computational and communication overhead of underwater nodes.

[0082] 4. Improve the reliability-latency-energy tradeoff: By explicitly introducing energy consumption, latency and delivery failure costs, the routing strategy no longer simply pursues a single indicator, but achieves a more balanced performance between delivery rate, end-to-end latency and energy consumption.

[0083] 5. Enhanced adaptability to dynamic environments: When node energy, queue load, link quality, and feasible topology change, the method can dynamically update the feasible next-hop set and adjust the route selection, thereby improving adaptability in complex underwater environments.

[0084] 6. Suitable for network expansion of different scales: The distributed routing decision module with shared parameters can be applied to multiple node scales, and the graph-enhanced value evaluation module during training can use topology information to enhance the policy generalization ability.

[0085] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0086] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the described functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0087] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0088] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0089] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0090] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0091] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0092] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A constraint-aware graph-enhanced routing decision method for underwater acoustic sensor networks, characterized in that, Includes the following steps: Obtain local observation information of the current forwarding node; Based on the local observation information, a multi-dimensional feasibility judgment is made on the candidate neighbor nodes to generate a set of feasible next hops; The local observation information and the feasible next-hop set are input into the distributed routing decision model to be trained, and the pre-trained routing actions are output. The pre-trained routing action is executed to obtain the execution result, and the local observation information is updated based on the execution result; Using the updated local observation information, a dynamic feasible graph is constructed, and graph augmentation value assessment is performed on the dynamic feasible graph to obtain a state value estimate; Using the state value estimation and the execution result, the distributed routing decision model to be trained is subjected to constraint-aware training to obtain the trained distributed routing decision model. The trained distributed routing decision model is deployed to an underwater node. The local observation information to be decided and the set of feasible next hops to be decided are input into the trained distributed routing decision model, and the target routing action is output.

2. The method of claim 1, wherein, The step of performing a multi-dimensional feasibility assessment on candidate neighbor nodes based on the local observation information and generating a feasible next-hop set includes the following steps: Based on the local observation information, extract the first remaining energy of the current forwarding node, the second remaining energy of the candidate neighbor node, the estimated single-hop delay of the candidate link, and the access history in the cumulative state of the current routing process; Determine whether the candidate neighbor node is within the communication coverage area of ​​the current forwarding node to obtain a first determination result; Determine whether the first remaining energy is greater than or equal to a preset transmission energy threshold to obtain a second determination result; Determine whether the second remaining energy is greater than or equal to a preset received energy threshold to obtain a third determination result; Determine whether the estimated single-hop delay is less than the preset single-hop delay upper limit to obtain the fourth determination result; Determine whether the candidate neighbor node exists in the access history to obtain the fifth determination result; The feasible next hop set is generated based on the first judgment result, the second judgment result, the third judgment result, the fourth judgment result, and the fifth judgment result.

3. The method of claim 1, wherein, The process of inputting the local observation information and the feasible next-hop set into the distributed routing decision model to be trained, and outputting pre-trained routing actions, includes the following steps: When the feasible next hop set is not empty, an action mask is generated based on the feasible next hop set; The local observation information is input into the feature encoding unit of the distributed routing decision model to be trained, and a candidate action feature vector is output; each component in the candidate action feature vector corresponds to each of the candidate neighbor nodes. Based on the action mask, infeasible candidate nodes in the candidate action feature vector are masked to obtain an action score vector; The action score vector is normalized to obtain the selection probability distribution of each feasible candidate node in the feasible next hop set; The feasible candidate node with the highest probability in the selection probability distribution is selected as the next hop node to obtain the pre-trained routing action.

4. The method of claim 1, wherein, The method further includes the following steps: When the set of feasible next hops is empty, activate the safe rollback action; In response to the security rollback action, the current routing process is terminated.

5. The method of claim 1, wherein, The process of constructing a dynamic feasible graph using the updated local observation information, performing graph augmentation value assessment on the dynamic feasible graph, and obtaining a state value estimate includes the following steps: Extract the state features of underwater nodes and sink nodes from the updated local observation information, and extract the feasible forwarding link relationships between underwater nodes and between underwater nodes and sink nodes; Based on the underwater node and the convergence node, a graph node set is constructed, and based on the feasible forwarding link relationship, a directed edge set is constructed. Based on the graph node set and the directed edge set, the dynamic feasible graph is obtained. Based on the state characteristics, the feasible forwarding link relationships, the first indication information of the current forwarding node, and the second indication information of the aggregation node, a graph augmentation value assessment is performed on the dynamic feasible graph to obtain the state value estimate.

6. The method of claim 1, wherein, The process of constrain-aware training of the distributed routing decision model to be trained, based on the state value estimation and the execution result, to obtain the trained distributed routing decision model, includes the following steps: Obtain the delivery failure cost, latency cost, and energy consumption cost from the execution result; Based on the first comparison result of the delivery failure cost and the first preset budget, the second comparison result of the delay cost and the second preset budget, and the third comparison result of the energy consumption cost and the third preset budget, the delivery reliability constraint corresponding to the delivery failure cost, the delay constraint corresponding to the delay cost, and the energy consumption constraint corresponding to the energy consumption cost are dynamically weighted and adjusted. Based on the state value estimation, the delivery reliability constraint after dynamic weight adjustment, the delay constraint after dynamic weight adjustment, and the energy consumption constraint after dynamic weight adjustment, the parameters of the distributed routing decision model to be trained are updated through adaptive constraint weights or Lagrange multiplier update methods to obtain the trained distributed routing decision model.

7. A constraint-aware graph-enhanced routing decision apparatus for underwater acoustic sensor networks, characterized in that, include: The network status acquisition module is used to obtain local observation information of the current forwarding node; The feasible link determination module is used to perform multi-dimensional feasibility determination on candidate neighbor nodes based on the local observation information and generate a set of feasible next hops; The action masking module is used to input the local observation information and the feasible next-hop set into the distributed routing decision model to be trained, and output pre-trained routing actions. The training phase route execution and state update module is used to execute the pre-trained route actions, obtain the execution results, and update the local observation information based on the execution results; The training period graph value assessment module is used to construct a dynamic feasible graph using the updated local observation information, perform graph augmentation value assessment on the dynamic feasible graph, and obtain a state value estimate. The constraint update module is used to perform constraint-aware training on the distributed routing decision model to be trained using the state value estimation and the execution result, so as to obtain the trained distributed routing decision model. The deployment module is used to deploy the trained distributed routing decision model to underwater nodes, input the local observation information to be decided and the set of feasible next hops to be decided into the trained distributed routing decision model, and output the target routing action.

8. An electronic device, comprising: Including the processor and memory; The memory is used to store programs; The processor executes the program to implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a program that is executed by a processor to implement the method as described in any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.