A clustering structure-oriented routing method and system for a flying ad hoc network

Abstract

The application provides a kind of cluster-oriented flight ad hoc network routing method and system, the steps of the method include: cluster organization is carried out to flight ad hoc network, for each node cluster, select cluster head node, the cluster head node is provided with cluster level decision unit;In each control period, based on the state data of all node clusters and network operating environment data, construct cross-layer observation vector;For each flight node, extract the cross-layer observation sub-vector related to the flight node from the cross-layer observation vector, input the cross-layer observation sub-vector into the cluster level decision unit corresponding to the flight node, obtain the routing action, the routing action includes routing mode information and relay candidate information routing action;Based on the routing action, update the routing table of the flight node;In the process of data transmission, each of the flight nodes performs routing transmission based on the routing table.

Description

Technical Field

[0001] This invention relates to the field of intelligent routing optimization technology, and in particular to a routing method and system for flight ad hoc networks with clustered structures. Background Technology

[0002] Ad hoc networks offer advantages such as rapid deployment, topology reconfigurability, and link self-organization in missions like emergency communication, power line inspection, disaster reconnaissance, and regional coverage. However, in complex mission environments, networked ad hoc nodes are typically in a state of continuous movement, with frequent topology changes, significant fluctuations in link quality, and limited remaining energy. When the network adopts a clustered forwarding structure, fluctuations occur in cross-cluster link quality, cluster head availability, and the local queue status of ad hoc nodes.

[0003] In existing technologies, the first type of solution mainly addresses network connectivity issues from the perspective of physical communication or control coordination, such as maintaining control stability by reconstructing communication topology, adjusting formation configuration, or building distributed observers; the second type of solution attempts to introduce reinforcement learning routing into flight ad hoc networks.

[0004] The first type of scheme usually focuses more on the recoverability of control signals or the reorganization of link geometry, and lacks fine-grained modeling for network layer route updates, link cost construction, route loop suppression and data service forwarding reachability; the second type of scheme relies only on single-layer observation information, such as location, distance or single link quality, and is difficult to reflect the joint state of the network layer, link layer, physical layer and energy layer at the same time.

[0005] Therefore, when existing networks adopt a clustered forwarding structure, it is difficult to solve the problems of original forwarding path failure, queue accumulation, route unreachability, and even loops caused by fluctuations in cross-cluster link quality, cluster head availability, and local queue status of flying nodes. Summary of the Invention

[0006] In view of this, embodiments of the present invention provide a flight ad hoc network routing method and system for clustered structures, in order to eliminate or improve one or more defects existing in the prior art.

[0007] One aspect of the present invention provides a routing method for flight ad hoc networks with clustered structures, the method comprising the following steps: The flight ad hoc network is clustered, and the flight nodes in the flight ad hoc network are divided into corresponding node clusters. For each node cluster, a cluster head node is selected, and the cluster head node is equipped with a cluster-level decision unit. In each control cycle, the cluster head node collects the status data and network operating environment data of its node cluster and publishes them to each cluster head node of the flight ad hoc network. Based on the status data and network operating environment data of all node clusters, a cross-layer observation vector is constructed. For each flight node, extract the cross-layer observation sub-vector related to the flight node from the cross-layer observation vector, and input the cross-layer observation sub-vector into the cluster-level decision unit corresponding to the flight node to obtain the routing action. The routing action includes the routing action of routing mode information and relay candidate information. The routing table of the flight node is updated based on the routing action; During data transmission, each of the flight nodes performs route transmission based on the routing table.

[0008] The above scheme first converts the global operating state, local state of each decision point, and inter-cluster connection relationship of the clustered ad hoc flight network into the current cross-layer observation. Each cluster-level routing decision unit outputs routing actions including routing mode information and flight node weight adjustment information. Then, the target routing table is determined based on the routing mode information, and the weight adjustment of each flight node is corrected according to the current network state and the forwarding results of the previous control cycle. The updated flight node weights are written into the routing table. Then, an executable route is formed based on the updated routing table and the candidate link state. The total edge cost contained in the current routing path is aggregated and normalized, and it is directly added to the initial reward function as a negative term. The next reward and the next cross-layer observation are calculated in combination with the actual data transmission results of the current control cycle, thereby driving the routing policy update of subsequent control cycles. This scheme can automatically adjust the routing table. When there are fluctuations in cross-cluster link quality, cluster head availability, and flight node local queue state, it can select the most suitable flight node for transmission based on the current network state, solving the problems of route unreachability and even loops.

[0009] In some embodiments of the present invention, the cluster-level decision unit of the cluster head node adopts a reinforcement learning model, and the method further includes the following steps: in the next control cycle of the current control cycle, calculating the edge cost of each edge in the node cluster based on the state data and network operating environment data of the node cluster where the cluster head node is located, and calculating the total edge cost, calculating the total reward function based on the total edge cost, and updating the network parameters of the reinforcement learning model of the cluster-level decision unit of the cluster head node based on the total reward function.

[0010] In some embodiments of the present invention, the state data of the node cluster includes the edge distance of each edge, and the network operating environment data includes the signal-to-noise ratio, interference intensity, and packet error rate. In the step of calculating the edge cost of each edge in the node cluster based on the state data of the node cluster where the cluster head node is located and the network operating environment data, and calculating the total edge cost: The basic edge cost of each edge is calculated based on the state data and network operating environment data of the node cluster to which the cluster head node belongs. The sum of the basic edge costs of each edge in the node cluster is then calculated as the total edge cost.

[0011] In some embodiments of the present invention, in the step of calculating the basic edge cost of each edge based on the state data of the node cluster to which the cluster head node belongs and the network operating environment data, the basic edge cost is calculated using the following formula: in, Indicates the flight node Point to flight node The basic edge cost, Indicates the flight node Point to flight node The normalized signal-to-noise ratio of the edge, Indicates the flight node Point to flight node The normalized edge distance of the edge. Indicates flight node The normalized interference intensity, Indicates the flight node Point to flight node The normalized packet error rate of the edges.

[0012] In some embodiments of the present invention, in the step of calculating the total reward function based on the total edge cost, an initial reward function is calculated based on the total edge cost, and a total reward function is calculated for each routing mode based on the initial reward function.

[0013] In some embodiments of the present invention, in the step of calculating the initial reward function based on the total edge cost, the reward weight value set for each routing mode is obtained, and the product of the calculated reward weight value for each routing mode and the total edge cost is used as the initial reward function for that routing mode.

[0014] In some embodiments of the present invention, in the step of calculating the total reward function for each routing mode based on the initial reward function, the reliability target component, delay target component, energy target component, and public safety constraint component are calculated based on the state data of the node cluster where the cluster head node is located and the network operating environment data; the sum of the reliability target component, delay target component, energy target component, and public safety constraint component is calculated and subtracted from the initial reward function to obtain the total reward function, and the total reward function is normalized to... Within the range.

[0015] In some embodiments of the present invention, the state data of the node cluster includes queue pressure and remaining energy ratio, and the network operating environment data includes packet delivery rate, end-to-end latency, and routing anomaly rate. The reliability target component, latency target component, energy target component, and public safety constraint component are calculated using the following formulas: in, Represents the reliability target component, Indicates the target delay component. Indicates the target energy component; This indicates the group delivery rate for the current control cycle. This represents the normalized average end-to-end delay of the current control cycle. This represents the normalized queue pressure of the node clusters traversed by the current route. This represents the average remaining energy ratio of the node clusters traversed by the current route. This indicates the routing anomaly rate for the current control cycle; , , and These represent the weight values ​​for the corresponding reliability, latency, energy, and public safety constraints, respectively.

[0016] Specifically, the group delivery rate is calculated using the following formula: in, This represents the total number of data packets sent by the source flying node during the current control cycle in the node cluster. This represents the number of data packets that successfully reached the destination flight node in the current control cycle within the node cluster; Normalized average end-to-end delay is calculated according to the following formula: ; in, This represents the set of data packets in the node cluster that successfully reached the destination flight node during the current control cycle; and These represent data packets in the node cluster. Sending and receiving times; This indicates the preset acceptable latency reference value. Indicates normalization to ; The average remaining energy ratio of a node cluster is calculated according to the following formula: in, Indicates the first Within the first control cycle, the first A set of flying nodes for a node cluster; Indicates the first The number of flying nodes in the set of flying nodes of a node cluster; Indicates flight node In the Remaining energy within each control cycle; Indicates flight node The preset reference energy.

[0017] The routing anomaly rate is calculated using the following formula: in, This indicates the number of data packets in the node cluster that experienced at least one of the following anomalies during the current control cycle: loop, no route, interface closed, or routing error.

[0018] In some embodiments of the present invention, each routing mode is provided with a corresponding routing table, the routing table recording the weight value corresponding to each flight node, the routing action recording the adjustment value of the weight value corresponding to each flight node, and the step of inputting the cross-layer observation sub-vector into the cluster-level decision unit corresponding to the flight node to obtain the routing action further includes performing security post-processing on the routing action output by the cluster-level decision unit, the security post-processing step including: The routing table corresponding to the routing action is determined based on the routing mode information corresponding to the routing action of the relay candidate information. Get the remaining energy ratio of the flying nodes in the current routing table. If the remaining energy ratio of the flying node is lower than the preset energy threshold, record the weight value of the flying node in the routing table and hide the flying node. The adjustment value corresponding to the flight node for hidden processing is determined from the routing action, and the adjustment value is set to zero.

[0019] In some embodiments of the present invention, the reinforcement learning model of the cluster-level decision-making unit includes a cluster-level individual action value network and a joint value hybrid network; The cluster-level individual action value network includes a state feature encoding branch, a topology feature encoding branch, a feature fusion layer, a routing pattern output branch, and a flight node weight output branch; The state feature encoding branch sequentially includes a first fully connected layer with 128 neurons, a ReLU activation layer, a second fully connected layer with 64 neurons, and a ReLU activation layer. The topological feature encoding branch sequentially includes a first graph convolutional layer with 32 output channels, a ReLU activation layer, a second graph convolutional layer with 64 output channels, and a masked average pooling layer. The feature fusion layer sequentially includes a feature splicing layer, a third fully connected layer with 128 neurons, a ReLU activation layer, a fourth fully connected layer with 64 neurons, and a ReLU activation layer. The routing mode output branch includes a fully connected output layer with three output flight nodes, which respectively output the action values ​​of link quality priority mode, cluster head priority mode, and energy priority mode. The flight node weight output branch includes a flight node feature splicing layer, a fifth fully connected layer with 64 neurons, a ReLU activation layer, and a shared fully connected output layer with 3 output flight nodes. The 3 output flight nodes respectively output the action value of reducing the flight node weight, maintaining the flight node weight, and increasing the flight node weight. The joint value hybrid network sequentially includes a non-negative weight generation layer, a hybrid hidden layer with 32 neurons, an ELU activation layer, and a single-flight node scalar output layer, wherein the single-flight node scalar output layer outputs the joint action value.

[0020] A second aspect of the present invention also provides a flight ad hoc network routing system for clustered structures. The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method described above.

[0021] A third aspect of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned flight ad hoc network routing method for clustered structures.

[0022] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the text, or may be learned by practice of the invention. The objects and other advantages of the invention will become apparent from the description and the accompanying drawings.

[0023] Those skilled in the art will understand that the objectives and advantages achievable with this invention are not limited to those specifically described above, and that the above and other objectives achievable with this invention will become clearer from the following detailed description. Attached Figure Description

[0024] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0025] Figure 1This is a schematic diagram illustrating one implementation of the flight ad hoc network routing method for clustered structures in this scheme; Figure 2 This is a schematic diagram illustrating another implementation of the flight ad hoc network routing method for clustered structures in this scheme. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0027] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0028] like Figure 1 As shown, this invention proposes a routing method for flight ad hoc networks with clustered structures. The steps of this method include: Step S100: The flight ad hoc network is clustered and organized by dividing the flight nodes in the flight ad hoc network into corresponding node clusters. For each node cluster, a cluster head node is selected, and the cluster head node is equipped with a cluster-level decision unit. Specifically, multiple flight nodes can be divided into multiple clusters based on the preset cluster size, existing task grouping results, or the clustering results already formed in the current flight ad hoc network; the cluster head node can be pre-selected by the staff.

[0029] Specifically, each flight node has preset relay candidate information, which is a set of relay node numbers for that flight node, and all relay nodes are flight nodes; each flight node has a routing table for each routing model, and the routing table records the weight value of each relay node.

[0030] In step S200, during each control cycle, the cluster head node collects the status data and network operating environment data of its node cluster and publishes them to each cluster head node of the flight ad hoc network. Based on the status data and network operating environment data of all node clusters, a cross-layer observation vector is constructed. Specifically, at the beginning of each control cycle, based on the network cross-layer state information formed by the data forwarding results of the previous control cycle, the global network state characteristics, the local state characteristics of each cluster-level routing decision point, and the inter-cluster adjacency matrix characteristics are obtained, and the current cross-layer observation is constructed.

[0031] Step S300: For each flight node, extract the cross-layer observation sub-vector related to the flight node from the cross-layer observation vector, and input the cross-layer observation sub-vector into the cluster-level decision unit corresponding to the flight node to obtain the routing action. The routing action includes the routing action of routing mode information and relay candidate information. Specifically, the cross-layer observation vector can be a vector constructed from physical layer information and link layer information. The physical layer information includes signal-to-noise ratio, interference intensity, and path loss; the link layer information includes bit error rate, packet error rate, and interface availability status.

[0032] Step S400: Update the routing table of the flight node based on the routing action; In the specific implementation process, multiple cluster-level decision units generate routing actions based on the current cross-layer observations, including routing mode information and node weight adjustment information. The routing mode information is used to determine the routing table corresponding to the current routing action, and the node weight adjustment information is used to increase or decrease the weight of each flying node in the corresponding routing table.

[0033] In step S500, during data transmission, each of the flying nodes performs route transmission based on the routing table.

[0034] In the specific implementation process, data transmission for the current control cycle is executed according to the updated routing table; after the data transmission is completed, the next reward and the next cross-layer observation are calculated based on the normalized total edge cost, routing mode information, node weight adjustment results and the execution results of the corresponding routes, forming state transition samples to update the routing policy.

[0035] Specifically, for each of the aforementioned flight nodes, the flight node with the largest weight value in the routing table can be selected as the subsequent transmission relay.

[0036] The above scheme first converts the global operating state, local state of each decision point, and inter-cluster connection relationship of the clustered ad hoc flight network into the current cross-layer observation. Each cluster-level routing decision unit outputs routing actions including routing mode information and flight node weight adjustment information. Then, the target routing table is determined based on the routing mode information, and the weight adjustment of each flight node is corrected according to the current network state and the forwarding results of the previous control cycle. The updated flight node weights are written into the routing table. Then, an executable route is formed based on the updated routing table and the candidate link state. The total edge cost contained in the current routing path is aggregated and normalized, and it is directly added to the initial reward function as a negative term. The next reward and the next cross-layer observation are calculated in combination with the actual data transmission results of the current control cycle, thereby driving the routing policy update of subsequent control cycles. This scheme can automatically adjust the routing table. When there are fluctuations in cross-cluster link quality, cluster head availability, and flight node local queue state, it can select the most suitable flight node for transmission based on the current network state, solving the problems of route unreachability and even loops.

[0037] like Figure 2 As shown, in some embodiments of the present invention, the cluster-level decision unit of the cluster head node adopts a reinforcement learning model. The method further includes the following steps: Step S600, in the next control cycle of the current control cycle, calculating the edge cost of each edge in the node cluster based on the state data and network operating environment data of the node cluster where the cluster head node is located, and calculating the total edge cost, calculating the total reward function based on the total edge cost, and updating the network parameters of the reinforcement learning model of the cluster-level decision unit of the cluster head node based on the total reward function.

[0038] In some embodiments of the present invention, the reinforcement learning model is trained based on the total reward function. This scheme can employ a dual-reward model. study; Specifically, The learning target value and the update formula can be expressed as: in, This represents the immediate reward corresponding to the state transition. Indicates the discount factor; Indicates the next state; Indicates the candidate action for the next state; Indicates the learning rate; This represents the timing difference error.

[0039] pair Learn to maintain the action value table separately and .

[0040] When updating In this case, the following can be used: renew Symmetrical processing is used at that time, by Select the next action and... Evaluate the selected actions. By separating action selection and action evaluation, overestimation caused by a single value estimate participating in both selection and evaluation can be reduced.

[0041] For deep double In this learning process, the online network is used to select the next action, and the target network is used to evaluate that action. In this scheme, the online network is a cluster-level individual action value network. in, The parameter is Online networks; Indicate the target network; Indicates a termination marker; Indicates smoothness loss. At that time, the target value only retains the immediate reward.

[0042] For joint action value learning with centralized training and distributed execution, the joint state transition sample is represented as: in, and These represent the combined cross-layer states before and after the action is executed; and These represent the local observation sets of all cluster-level routing decision units before and after the action is executed; Indicates a joint action; This represents shared rewards. Let each decision-making unit's individual action value for the selected action be composed of a vector. The value of a combined action can be expressed as: in, and Indicates based on joint cross-layer state The generated mixed weights, and This indicates the corresponding bias. This represents the exponential linear unit activation function. During implementation, it will be... and By restricting the values ​​to non-negative values, the value of the joint action is kept monotonic with respect to the value of each individual action, thereby supporting each decision-making unit to independently select the optimal individual action during the distributed execution phase.

[0043] In some embodiments of the present invention, the state data of the node cluster includes the edge distance of each edge, and the network operating environment data includes signal-to-noise ratio, interference intensity, and packet error rate. In the step of calculating the edge cost of each edge in the node cluster based on the state data of the node cluster where the cluster head node is located and the network operating environment data, and calculating the total edge cost: The basic edge cost of each edge is calculated based on the state data of the node cluster to which the cluster head node belongs and the network operating environment data. The sum of the basic edge costs of each edge in the node cluster is then calculated as the total edge cost.

[0044] In some embodiments of the present invention, in the step of calculating the basic edge cost of each edge based on the state data of the node cluster to which the cluster head node belongs and the network operating environment data, the basic edge cost is calculated using the following formula: in, Indicates the flight node Point to flight node The basic edge cost, Indicates the flight node Point to flight node The normalized signal-to-noise ratio of the edge, Indicates the flight node Point to flight node The normalized edge distance of the edge. Indicates flight node The normalized interference intensity, Indicates the flight node Point to flight node The normalized packet error rate of the edges.

[0045] In some embodiments of the present invention, in the step of calculating the total reward function based on the total edge cost, an initial reward function is calculated based on the total edge cost, and a total reward function is calculated for each routing mode based on the initial reward function.

[0046] In some embodiments of the present invention, in the step of calculating the initial reward function based on the total edge cost, the reward weight value set for each routing mode is obtained, and the product of the calculated reward weight value for each routing mode and the total edge cost is used as the initial reward function for that routing mode.

[0047] In some embodiments of the present invention, in the step of calculating the total reward function for each routing mode based on the initial reward function, the reliability target component, the delay target component, the energy target component, and the public safety constraint component are calculated based on the state data of the node cluster where the cluster head node is located and the network operating environment data; the sum of the reliability target component, the delay target component, the energy target component, and the public safety constraint component is calculated and subtracted from the initial reward function to obtain the total reward function.

[0048] In some embodiments of the present invention, the state data of the node cluster includes queue pressure and remaining energy ratio, and the network operating environment data includes packet delivery rate, end-to-end latency, and routing anomaly rate. The reliability target component, latency target component, energy target component, and public safety constraint component are calculated using the following formulas: in, Represents the reliability target component, Indicates the target delay component. Indicates the target energy component; This indicates the group delivery rate for the current control cycle. This represents the normalized average end-to-end delay of the current control cycle. This represents the normalized queue pressure of the node clusters traversed by the current route. This represents the average remaining energy ratio of the node clusters traversed by the current route. This indicates the routing anomaly rate for the current control cycle; , , and These represent the weight values ​​for the corresponding reliability, latency, energy, and public safety constraints, respectively. + + + .

[0049] Specifically, the group delivery rate is calculated using the following formula: in, This represents the total number of data packets sent by the source flying node during the current control cycle in the node cluster. This represents the number of data packets that successfully reached the destination flight node in the current control cycle within the node cluster; Normalized average end-to-end delay is calculated according to the following formula: ; in, This represents the set of data packets in the node cluster that successfully reached the destination flight node during the current control cycle; and These represent data packets in the node cluster. Sending and receiving times; This indicates the preset acceptable latency reference value. Indicates normalization to ; The average remaining energy ratio of a node cluster is calculated according to the following formula: in, Indicates the first Within the first control cycle, the first A set of flying nodes for a node cluster; Indicates the first The number of flying nodes in the set of flying nodes of a node cluster; Indicates flight node In the Remaining energy within each control cycle; Indicates flight node The preset reference energy.

[0050] The routing anomaly rate is calculated using the following formula: in, This indicates the number of data packets in the node cluster that experienced at least one of the following anomalies during the current control cycle: loop, no route, interface closed, or routing error.

[0051] In some embodiments of the present invention, each routing mode is provided with a corresponding routing table, the routing table recording the weight value corresponding to each flight node, the routing action recording the adjustment value of the weight value corresponding to each flight node, and the step of inputting the cross-layer observation sub-vector into the cluster-level decision unit corresponding to the flight node to obtain the routing action further includes performing security post-processing on the routing action output by the cluster-level decision unit, the security post-processing step including: The routing table corresponding to the routing action is determined based on the routing mode information corresponding to the routing action of the relay candidate information. Get the remaining energy ratio of the flying nodes in the current routing table. If the remaining energy ratio of the flying node is lower than the preset energy threshold, record the weight value of the flying node in the routing table and hide the flying node. The adjustment value corresponding to the flight node for hidden processing is determined from the routing action, and the adjustment value is set to zero.

[0052] In some embodiments of the present invention, the reinforcement learning model of the cluster-level decision-making unit includes a cluster-level individual action value network and a joint value hybrid network; The cluster-level individual action value network includes a state feature encoding branch, a topology feature encoding branch, a feature fusion layer, a routing pattern output branch, and a flight node weight output branch; The state feature encoding branch sequentially includes a first fully connected layer with 128 neurons, a ReLU activation layer, a second fully connected layer with 64 neurons, and a ReLU activation layer. The topological feature encoding branch sequentially includes a first graph convolutional layer with 32 output channels, a ReLU activation layer, a second graph convolutional layer with 64 output channels, and a masked average pooling layer. The feature fusion layer sequentially includes a feature splicing layer, a third fully connected layer with 128 neurons, a ReLU activation layer, a fourth fully connected layer with 64 neurons, and a ReLU activation layer. The routing mode output branch includes a fully connected output layer with three output flight nodes, which respectively output the action values ​​of link quality priority mode, cluster head priority mode, and energy priority mode. The flight node weight output branch includes a flight node feature splicing layer, a fifth fully connected layer with 64 neurons, a ReLU activation layer, and a shared fully connected output layer with 3 output flight nodes. The 3 output flight nodes respectively output the action value of reducing the flight node weight, maintaining the flight node weight, and increasing the flight node weight. The joint value hybrid network sequentially includes a non-negative weight generation layer, a hybrid hidden layer with 32 neurons, an ELU activation layer, and a single-flight node scalar output layer, wherein the single-flight node scalar output layer outputs the joint action value.

[0053] Specifically, the reinforcement learning model includes an online network group and a target network group; both the online network group and the target network group include a cluster-level individual action value network and a joint value mixing network with the same structure; the joint value mixing network of the online network group mixes the individual action values ​​of each online cluster and outputs the current joint action value; the joint value mixing network of the target network group mixes the individual action values ​​of each target cluster and outputs the target joint action value; a training loss is determined based on the current joint action value and the target joint action value, and the network parameters of the online network group are updated based on the training loss; the network parameters of the target network group are periodically updated based on the network parameters of the online network group.

[0054] Using the above scheme, state features and inter-cluster topology features are extracted and fused through different branches, enabling the model to simultaneously retain the node cluster operating state and inter-cluster connection relationship; the routing mode output branch and the node weight output branch correspond to the selection of the target routing table and the increase or decrease of the weight of each node, respectively; the joint value hybrid network coordinates multiple cluster-level decision units during the centralized training phase, and retains the ability of each cluster-level decision unit to independently generate routing actions based on local observations during the execution phase.

[0055] This invention also provides a flight ad hoc network routing system for clustered structures. The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method described above.

[0056] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned flight ad hoc network routing method for clustered structures. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0057] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0058] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0059] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0060] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A routing method for ad hoc flight networks with clustered structures, characterized in that, The steps of this method include: The flight ad hoc network is clustered, and the flight nodes in the flight ad hoc network are divided into corresponding node clusters. For each node cluster, a cluster head node is selected, and the cluster head node is equipped with a cluster-level decision unit. In each control cycle, the cluster head node collects the status data and network operating environment data of its node cluster and publishes them to each cluster head node of the flight ad hoc network. Based on the status data and network operating environment data of all node clusters, a cross-layer observation vector is constructed. For each flight node, extract the cross-layer observation sub-vector related to the flight node from the cross-layer observation vector, and input the cross-layer observation sub-vector into the cluster-level decision unit corresponding to the flight node to obtain the routing action. The routing action includes the routing action of routing mode information and relay candidate information. The routing table of the flight node is updated based on the routing action; During data transmission, each of the flight nodes performs route transmission based on the routing table.

2. The flight ad hoc network routing method for clustered structures according to claim 1, characterized in that, The cluster-level decision unit of the cluster head node adopts a reinforcement learning model. The method further includes the following steps: in the next control cycle of the current control cycle, calculating the edge cost of each edge in the node cluster based on the state data and network operating environment data of the node cluster where the cluster head node is located, and calculating the total edge cost; calculating the total reward function based on the total edge cost; and updating the network parameters of the reinforcement learning model of the cluster-level decision unit of the cluster head node based on the total reward function.

3. The flight ad hoc network routing method for clustered structures according to claim 2, characterized in that, The state data of the node cluster includes the edge distance of each edge, and the network operating environment data includes the signal-to-noise ratio, interference intensity, and packet error rate. In the steps of calculating the edge cost of each edge in the node cluster based on the state data of the node cluster where the cluster head node belongs and the network operating environment data, and calculating the total edge cost: Based on the state data and network operating environment data of the node cluster to which the cluster head node belongs, the basic edge cost of each edge is calculated, and the sum of the basic edge costs of each edge in the node cluster is calculated as the total edge cost.

4. The flight ad hoc network routing method for clustered structures according to claim 2, characterized in that, In the step of calculating the basic edge cost of each edge based on the state data of the cluster where the cluster head node belongs and the network operating environment data, the basic edge cost is calculated using the following formula: in, Indicates the flight node Point to flight node The basic edge cost, Indicates the flight node Point to flight node The normalized signal-to-noise ratio of the edge, Indicates the flight node Point to flight node The normalized edge distance of the edge. Indicates flight node The normalized interference intensity, Indicates the flight node Point to flight node The normalized packet error rate of the edges.

5. The flight ad hoc network routing method for clustered structures according to claim 4, characterized in that, In the step of calculating the total reward function based on the total edge cost, an initial reward function is calculated based on the total edge cost, and a total reward function is calculated for each routing mode based on the initial reward function.

6. The flight ad hoc network routing method for clustered structures according to claim 5, characterized in that, In the step of calculating the initial reward function based on the total edge cost, the reward weight value set for each routing mode is obtained, and the product of the calculated reward weight value for each routing mode and the total edge cost is used as the initial reward function for that routing mode. In the step of calculating the total reward function for each routing mode based on the initial reward function, the reliability target component, the delay target component, the energy target component, and the public safety constraint component are calculated based on the state data of the node cluster where the cluster head node is located and the network operating environment data. The sum of the reliability target component, delay target component, energy target component, and public safety constraint component is calculated and subtracted from the initial reward function to obtain the total reward function.

7. The flight ad hoc network routing method for clustered structures according to claim 6, characterized in that, The node cluster status data includes queue pressure and remaining energy ratio, and the network operating environment data includes packet delivery rate, end-to-end latency, and routing anomaly rate. The reliability target component, latency target component, energy target component, and public safety constraint component are calculated using the following formulas: in, Represents the reliability target component, Indicates the target delay component. Indicates the target energy component; This indicates the group delivery rate for the current control cycle. This represents the normalized average end-to-end delay of the current control cycle. This represents the normalized queue pressure of the node clusters traversed by the current route. This represents the average remaining energy ratio of the node clusters traversed by the current route. This indicates the routing anomaly rate for the current control cycle; , , and These represent the weight values ​​for the corresponding reliability, latency, energy, and public safety constraints, respectively.

8. The flight ad hoc network routing method for clustered structures according to claim 1, characterized in that, Each routing mode has a corresponding routing table, which records the weight value of each flight node. The routing action records the adjustment value of the weight value of each flight node. The step of inputting the cross-layer observation sub-vector into the cluster-level decision unit corresponding to the flight node to obtain the routing action further includes performing security post-processing on the routing action output by the cluster-level decision unit. The security post-processing steps include: The routing table corresponding to the routing action is determined based on the routing mode information corresponding to the routing action of the relay candidate information. Get the remaining energy ratio of the flying nodes in the current routing table. If the remaining energy ratio of the flying node is lower than the preset energy threshold, record the weight value of the flying node in the routing table and hide the flying node. The adjustment value corresponding to the flight node for hidden processing is determined from the routing action, and the adjustment value is set to zero.

9. The flight ad hoc network routing method for clustered structures according to claim 2, characterized in that, The reinforcement learning model of the cluster-level decision-making unit includes a cluster-level individual action value network and a joint value hybrid network; The cluster-level individual action value network includes a state feature encoding branch, a topology feature encoding branch, a feature fusion layer, a routing pattern output branch, and a flight node weight output branch; The state feature encoding branch sequentially includes a first fully connected layer with 128 neurons, a ReLU activation layer, a second fully connected layer with 64 neurons, and a ReLU activation layer. The topological feature encoding branch sequentially includes a first graph convolutional layer with 32 output channels, a ReLU activation layer, a second graph convolutional layer with 64 output channels, and a masked average pooling layer. The feature fusion layer sequentially includes a feature splicing layer, a third fully connected layer with 128 neurons, a ReLU activation layer, a fourth fully connected layer with 64 neurons, and a ReLU activation layer. The routing mode output branch includes a fully connected output layer with three output flight nodes, which respectively output the action values ​​of link quality priority mode, cluster head priority mode, and energy priority mode. The flight node weight output branch includes a flight node feature splicing layer, a fifth fully connected layer with 64 neurons, a ReLU activation layer, and a shared fully connected output layer with 3 output flight nodes. The 3 output flight nodes respectively output the action value of reducing the flight node weight, maintaining the flight node weight, and increasing the flight node weight. The joint value hybrid network sequentially includes a non-negative weight generation layer, a hybrid hidden layer with 32 neurons, an ELU activation layer, and a single-flight node scalar output layer, wherein the single-flight node scalar output layer outputs the joint action value.

10. A flight ad hoc network routing system for clustered structures, characterized in that, The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method as described in any one of claims 1 to 9.

Images