Ad hoc network routing method based on neighborhood topological graph representation
By using neighborhood topology graph representation and graph neural network reinforcement learning framework, the problem of topology changes caused by high-speed node movement in UAV ad hoc networks is solved, achieving adaptive and efficient routing decisions and improving network performance.
Patent Information
- Application Number
- CN202511504609.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-21
- Publication Date
- 2026-01-16
AI Technical Summary
In UAV ad hoc networks, the high-speed movement of nodes leads to frequent changes in network topology and an increased probability of link interruption. Traditional reinforcement learning frameworks struggle to adapt to variable-length inputs, neglecting the relative positions of neighboring nodes and topology information, thus affecting the accuracy and performance of routing decisions.
A neighborhood topology graph-based approach is adopted, which obtains neighbor state information by periodically broadcasting HELLO messages by nodes, constructs vertex and edge feature vectors, and dynamically decides the next-hop route using a graph neural network reinforcement learning framework, and uses the value function of neighbor nodes for route selection.
It achieves adaptive and efficient routing for self-organizing networks in high-mobility environments, effectively utilizing neighborhood topology information for distributed routing decisions, thereby improving network throughput and latency performance.
Smart Images
Figure CN121357633A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of communication technology, specifically to a self-organizing network routing method based on neighborhood topology graph representation. Background Technology
[0002] In recent years, with the development of information technology, unmanned aerial vehicle (UAV) ad hoc networks have been widely used in smart cities, modern warfare, and emergency rescue, serving as the core of information sharing in ad hoc networks. However, the high-speed movement of UAV nodes leads to frequent changes in network topology and a significantly increased probability of link interruptions, severely affecting the accuracy of routing decisions and reducing network throughput and latency performance.
[0003] Reinforcement learning-based routing schemes optimize the routing decision-making process through the interaction between agents and the environment, effectively addressing the problem of degraded routing performance under highly dynamic conditions. However, since the number of neighbors for different nodes is not fixed, traditional reinforcement learning frameworks based on linear layers struggle to adapt to variable-length inputs. Furthermore, the linear representation of the neighborhood topology tends to overlook the relative positions and topological information of neighboring nodes, resulting in an inability to capture network topology features and thus limiting routing performance. Reinforcement learning frameworks based on neighborhood topology graph representation and incorporating graph neural networks can effectively overcome these shortcomings. Summary of the Invention
[0004] To address the shortcomings of existing technologies, this invention provides a self-organizing network routing method based on neighborhood topology graph representation. It solves the problem of how to represent the one-hop neighborhood state of each node in a UAV self-organizing network using a topology graph and combine graph neural network reinforcement learning to dynamically decide the next hop, thereby achieving adaptive and efficient routing in high-mobility environments.
[0005] To achieve the above objectives, the present invention provides the following technical solution: a self-organizing network routing method based on neighborhood topology graph representation, comprising: Each node periodically obtains its own status parameters through sensing devices and broadcasts HELLO messages to neighboring nodes within a one-hop range; The routing decision node receives the HELLO message from the neighbor node and extracts the state information of the neighbor node and caches it in the local neighbor table; When the routing decision node generates a routing requirement, the routing decision node constructs a vertex feature vector for itself based on the local neighbor table information; Based on the local neighbor table information, vertex feature vectors are constructed for the neighbor nodes; Based on the local neighbor table information, edge feature vectors are constructed for the topological relationship between the routing decision node and the neighbor nodes; Based on the local neighbor table information, edge feature vectors are constructed for the topological relationships of neighboring nodes within each other's communication range; The obtained vertex feature vectors and edge feature vectors are used to construct a neighborhood topology graph, which is then input into a reinforcement learning framework built by a graph neural network to calculate the value function of the neighboring nodes. The routing decision node selects the neighbor node with the largest value function value as the next hop and delivers the data packet.
[0006] Preferably, the self-state parameters include position, velocity, remaining energy, MAC layer buffer queue, and communication link quality, and the node reduces the measurement error of the sensing device through filtering operations.
[0007] Preferably, the status information includes location, speed, remaining energy, MAC layer cache queue length, and signal-to-interference-plus-noise ratio (SINNR). The SINNR is the ratio of the signal power to the sum of noise power and interference power of data packets received by the neighboring node from other neighboring nodes. The routing decision node caches the status information in the local neighbor table and sets an expiration date for each entry. If an entry expires, it is removed. If a neighboring node receives a new HELLO message within the expiration date, the corresponding entry is updated and the expiration date is reset.
[0008] Preferably, when the routing decision node generates a routing request, it constructs a corresponding vertex feature vector for itself based on the relative position and relative speed with the destination node in the local neighbor table.
[0009] Preferably, for each neighbor node in the local neighbor table, a vertex feature vector of the neighbor node is constructed based on the relative position, relative speed, remaining energy, and MAC layer cache queue length of the neighbor node and the routing destination node.
[0010] Preferably, an edge feature vector of [decision node, neighbor node] is constructed for the topological relationship between the routing decision node and each neighbor node in the local neighbor table. The edge feature vector is composed of the relative position, relative speed and link signal-to-interference-plus-noise ratio of the routing decision node and each neighbor node.
[0011] Preferably, if any two neighboring nodes in the local neighbor table are within one-hop communication range of each other, then an edge feature vector of [neighboring node, neighboring node] is constructed, wherein the edge feature vector is composed of the relative position, relative speed and link signal-to-interference-plus-noise ratio of the two neighboring nodes.
[0012] Preferably, the routing decision node selects the node with the largest value function as the next hop according to the value function of each neighbor node output by the graph reinforcement learning framework, using a greedy strategy and the following model formula: in, This represents the set of one-hop neighbor nodes of the routing decision node. ) indicates at time The neighborhood topology graph after characterization. For the set of vertices, Indicates selecting neighbor nodes As the next jump action. Indicates the state Take action below The subsequent value function estimation, The node that is ultimately selected as the next hop for receiving data packets.
[0013] This invention provides a routing method for ad hoc networks based on neighborhood topology graph representation. It offers the following advantages: it provides a distributed neighborhood topology graph representation method, allowing routing decision nodes to complete graph representation based on neighbor information within a one-hop communication range.
[0014] This invention not only considers the topological relationship between the decision node and its neighboring nodes, but also the topological relationship between one-hop neighbors. Combined with the message propagation mechanism of graph neural networks, it enables the use of multi-hop neighborhood topological information, including one-hop neighborhood topology, during routing decisions, which is beneficial for reinforcement learning algorithms to learn better routing strategies. Attached Figure Description
[0015] Figure 1 This is a flowchart illustrating the process of realizing the invention; Figure 2 This is a schematic diagram of the one-hop neighborhood topology graph representation process; Figure 3 This is a schematic diagram of the neighborhood topology graph representation and graph reinforcement learning routing decision process; Figure 4 This is a schematic diagram illustrating the node status information for implementing the present invention. Detailed Implementation
[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0017] Example 1 like Figure 1-4As shown, this embodiment of the invention provides a self-organizing network routing method based on neighborhood topology graph representation. The method includes each node periodically obtaining its own state parameters through a sensing device and broadcasting a HELLO message to neighboring nodes within a one-hop range. The state parameters include location, speed, remaining energy, MAC layer buffer queue, and communication link quality. Nodes reduce measurement errors of the sensing device through filtering operations.
[0018] The routing decision node receives HELLO messages from neighboring nodes and extracts their state information, which is then cached in its local neighbor table. This state information includes location, speed, remaining energy, MAC layer buffer queue length, and signal-to-interference-plus-noise ratio (SINNR). SINNR is the ratio of the signal power to the sum of noise power and interference power of data packets received by a neighboring node from other neighboring nodes. The routing decision node caches this state information in its local neighbor table and sets an expiration date for each entry. Expired entries are removed. If a neighboring node receives a new HELLO message within the expiration period, the corresponding entry is updated, and its expiration date is reset.
[0019] The specific implementation method is as follows: If a routing decision node receives a HELLO message broadcast by a neighbor node, it will parse the HELLO message, extract the neighbor state information and cache it in the local neighbor table, and calculate the validity period of the neighbor table entry based on the movement status of the decision node and the neighbor node.
[0020] When a routing decision node generates a routing request, it constructs a vertex feature vector for itself based on its local neighbor table information. Specifically, it constructs a corresponding vertex feature vector for itself based on its relative position and relative speed with the destination node in its local neighbor table.
[0021] The specific implementation method is as follows: When decision node When a routing requirement arises, It is necessary to combine the neighbor table to perform graph representation of its one-hop neighborhood topology. It can be modeled as vertices in a graph structure. The corresponding vertex feature vector is derived from With the destination node of the route The relative position and relative velocity constitute, that is ,in pass The motion state is measured by sensors such as GPS and gyroscopes, and the results are obtained after appropriate filtering. It can be obtained through location service systems, therefore The corresponding vertex feature vector can be derived from Distributed construction.
[0022] neighbor nodes For example, It can be modeled as vertices in a graph structure, and its vertex feature vectors are derived from... With the destination node of the route Relative position, relative velocity, Remaining battery energy The length of the MAC layer cache queue is constituted, i.e. All of the above parameters can be obtained from the decision node. Obtained through local neighbor tables and location service systems, therefore The corresponding vertex feature vector can be derived from Distributed construction. (Among them) They represent The maximum battery level and the maximum length of the MAC layer cache queue in the initial state. This indicates a normalization operation.
[0023] neighbor nodes For example, With decision nodes The topological relationships between nodes can be modeled as edges in a graph structure, where edge < decision node. Neighbor nodes The eigenvectors of > are composed of and Relative position, relative speed, link Signal-to-interference-plus-noise ratio, link The signal-to-interference-plus-noise ratio (SIR) is composed of, i.e.: { , , , }
[0024] All of the above parameters can be obtained from the decision node. Obtained through the local neighbor table, therefore the edge < decision node Neighbor nodes The corresponding feature vector can be derived from... Distributed construction. Decision nodes. Following the steps above, traverse all neighbor nodes in the local neighbor table and construct edge feature vectors for all neighbor nodes <decision node, neighbor node>.
[0025] Based on the local neighbor table information, vertex feature vectors are constructed for neighbor nodes. For each neighbor node in the local neighbor table, a vertex feature vector is constructed based on the relative position, relative speed, remaining energy, and MAC layer cache queue length of the neighbor node and the routing destination node.
[0026] Based on the local neighbor table information, edge feature vectors are constructed for the topological relationship between the routing decision node and its neighbor nodes. An edge feature vector of [decision node, neighbor node] is constructed for the topological relationship between the routing decision node and each neighbor node in the local neighbor table. The edge feature vector consists of the relative position, relative speed, and link signal-to-interference-plus-noise ratio of the routing decision node and each neighbor node.
[0027] Based on the local neighbor table information, edge feature vectors are constructed for the topological relationships of neighbor nodes within each other's communication range. If any two neighbor nodes in the local neighbor table are within each other's one-hop communication range, then an edge feature vector of [neighbor node, neighbor node] is constructed. The edge feature vector consists of the relative position, relative speed, and link signal-to-interference-plus-noise ratio of the two neighbor nodes.
[0028] The specific implementation method is as follows: Decision Node The local neighbor table can be used to determine whether any two neighbor nodes are within each other's one-hop communication range. and For example, if the two are within each other's one-hop communication range, that is... , For communication range, the topological relationship between the two can be modeled as edges in a graph structure, where an edge < a neighboring node. Neighbor nodes The feature vector is composed of and Relative position, relative speed, link Signal-to-interference-plus-noise ratio, link The signal-to-interference-plus-noise ratio (SIR) is composed of, i.e.: All of the above parameters can be obtained from the decision node. Obtained through the local neighbor table, therefore the edge < neighbor node Neighbor nodes The corresponding feature vector can be derived from... Distributed construction. Decision nodes. Following the steps above, traverse all neighbor node pairs in the local neighbor table, and construct the <neighbor node, neighbor node> edge feature vector for all neighbor node pairs that satisfy the one-hop communication range constraint.
[0029] The obtained vertex feature vectors and edge feature vectors are combined to form a neighborhood topology graph, which is then input into a reinforcement learning framework built from a graph neural network to calculate the value function of neighboring nodes.
[0030] The routing decision node selects the neighbor node with the highest value function value as the next hop and delivers the data packet.
[0031] Based on the value functions of each neighbor node output by the graph reinforcement learning framework, the routing decision node selects the node with the largest value function as the next hop using a greedy strategy according to the following model formula: in, This represents the set of one-hop neighbor nodes of the routing decision node. ) indicates at time The neighborhood topology graph after characterization. For the set of vertices, Indicates selecting neighbor nodes As the next jump action. Indicates the state Take action below The subsequent value function estimation, The node that is ultimately selected as the next hop for receiving data packets.
[0032] The specific implementation method is as follows: Figure 2 The diagram shows a schematic representation of the one-hop neighborhood topology graph designed in this invention. As a routing decision node, All A neighbor who jumped. By receiving HELLO messages from neighboring nodes, the system obtains the neighboring node's ID, remaining battery percentage, MAC queue status, location, speed, and link quality status, and caches this information in its local neighbor table. The corresponding neighbor entry can be denoted as .when When a routing requirement arises, It is necessary to combine the local neighbor table information to determine whether any two neighbor nodes are within each other's one-hop communication range, and to perform graph representation of the one-hop neighborhood topology, setting corresponding vertex feature vectors and edge feature vectors. For example, the vertex feature vector corresponding to the decision node depends on the node's position and speed; the vertex feature vector corresponding to the neighbor node depends on the node's position, speed, MAC queue status, and remaining battery percentage; the feature vectors of the edge <decision node, neighbor node> and <neighbor node, neighbor node> depend on the relative position, relative speed, and bidirectional link signal-to-interference-plus-noise ratio of the two nodes connected by the edge.
[0033] Example 2 Figure 3The diagram illustrates the neighborhood topology graph representation and graph reinforcement learning routing decision-making process designed in this invention. Combining the HELLO message interaction mechanism between nodes, neighbor relationship representation, and routing table maintenance and online model updates, the overall process can be divided into three steps: The first step is node neighborhood awareness. Nodes broadcast HELLO messages containing location, speed, remaining energy, MAC layer cache queue length, and link quality at preset intervals, and receive HELLO messages from neighbors within a one-hop range. They extract neighbor status information by parsing the messages and cache it in their local neighbor table. Simultaneously, they record the reception time and dynamically set the validity period of the entry based on the relative movement speed between nodes and link stability. Expired entries are periodically cleaned up, and the table is refreshed upon receiving an update message to ensure the freshness and reliability of the neighbor table. The second step is routing request processing. When a node receives a routing request, it first checks its local routing table. If a destination node-next-hop mapping that has not expired exists, it directly returns a routing response and resets the expiration time of the routing table entry. Otherwise, the node constructs the current next hop based on the latest neighbor table information. The process involves inputting a neighborhood topology graph into a trained or online-updated graph reinforcement learning model. The model infers the value function of each candidate neighbor node and selects the highest-valued node as the next hop based on a greedy strategy. Simultaneously, the routing discovery result is written to the routing table with an expiration time consistent with the neighbor table entries, ensuring rapid response to subsequent identical requests. The third step is model training. When the reinforcement learning model's performance falls below a preset threshold or the network environment changes significantly, the routing decision node automatically enters training mode. It employs an exploratory strategy to select the next hop and obtains immediate rewards and neighborhood topology information based on link feedback. The next-hop node then transmits back a state-action-reward-next-state interaction experience quadruple and caches it in an experience pool. The model periodically samples randomly from the experience pool to update parameters and synchronize with the target network. After convergence based on training error and online test results, it exits training mode, thus achieving adaptive learning and robust routing decision-making for highly dynamic topologies.
[0034] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A routing method for ad hoc networks based on neighborhood topology graph representation, characterized in that, include: Each node periodically obtains its own status parameters through sensing devices and broadcasts HELLO messages to neighboring nodes within a one-hop range; The routing decision node receives the HELLO message from the neighbor node and extracts the state information of the neighbor node and caches it in the local neighbor table; When the routing decision node generates a routing requirement, the routing decision node constructs a vertex feature vector for itself based on the local neighbor table information; Based on the local neighbor table information, vertex feature vectors are constructed for the neighbor nodes; Based on the local neighbor table information, edge feature vectors are constructed for the topological relationship between the routing decision node and the neighbor nodes; Based on the local neighbor table information, edge feature vectors are constructed for the topological relationships of neighboring nodes within each other's communication range; The obtained vertex feature vectors and edge feature vectors are used to construct a neighborhood topology graph. The graph is then input into a reinforcement learning framework built from a graph neural network to calculate the value function of the neighboring nodes. The routing decision node selects the neighbor node with the largest value function value as the next hop and delivers the data packet.
2. The ad hoc network routing method based on neighborhood topology graph representation according to claim 1, characterized in that: The self-state parameters include position, velocity, remaining energy, MAC layer buffer queue, and communication link quality. The node reduces the measurement error of the sensing device through filtering operations.
3. The ad hoc network routing method based on neighborhood topology graph representation according to claim 1, characterized in that: The status information includes location, speed, remaining energy, MAC layer cache queue length, and signal-to-interference-plus-noise ratio (SIR). The SIR is the ratio of the signal power to the sum of noise power and interference power of data packets received by the neighboring node from other neighboring nodes. The routing decision node caches the status information in the local neighbor table and sets an expiration date for each entry. If an entry expires, it is removed. If a neighboring node receives a new HELLO message within the expiration date, the corresponding entry is updated and the expiration date is reset.
4. The ad hoc network routing method based on neighborhood topology graph representation according to claim 1, characterized in that: When the routing decision node generates a routing request, it constructs a corresponding vertex feature vector for itself based on the relative position and relative speed with the destination node in the local neighbor table.
5. The ad hoc network routing method based on neighborhood topology graph representation according to claim 1, characterized in that: For each neighbor node in the local neighbor table, a vertex feature vector of the neighbor node is constructed based on the relative position, relative speed, remaining energy, and MAC layer cache queue length of the neighbor node and the routing destination node.
6. The ad hoc network routing method based on neighborhood topology graph representation according to claim 1, characterized in that: For the topological relationship between the routing decision node and each neighbor node in the local neighbor table, construct an edge feature vector of [decision node, neighbor node]. The edge feature vector is composed of the relative position, relative speed and link signal-to-interference-plus-noise ratio of the routing decision node and each neighbor node.
7. The ad hoc network routing method based on neighborhood topology graph representation according to claim 1, characterized in that: If any two neighboring nodes in the local neighbor table are within one-hop communication range of each other, then an edge feature vector of [neighboring node, neighboring node] is constructed. The edge feature vector is composed of the relative position, relative speed and link signal-to-interference-plus-noise ratio of the two neighboring nodes.
8. The ad hoc network routing method based on neighborhood topology graph representation according to claim 1, characterized in that: The routing decision node selects the node with the largest value function as the next hop according to the following model formula, based on the value function of each neighbor node output by the graph reinforcement learning framework: in, This represents the set of one-hop neighbor nodes of the routing decision node. ) indicates at time The neighborhood topology graph after characterization. For the set of vertices, Represents the set of routing decision nodes. Let be the set of edges. Let dv be the vertex feature matrix, and dv be the length of the vertex eigenvector. Let de be the edge feature matrix, and de be the length of the edge feature vector. Indicates selecting neighbor nodes As the next jump action. Indicates the state Take action below The subsequent value function estimation, The node that is ultimately selected as the next hop for receiving data packets.