A cross-layer routing method for high-density flight ad hoc networks

Abstract

The application belongs to the technical field of unmanned aerial vehicle ad hoc network communication and intelligent routing, and discloses a cross-layer routing method for high-density flight ad hoc network. One-hop neighbor state information of a current forwarding node is input; a local geometric filtering area is constructed according to the spatial direction of the current node and a destination node, candidate neighbor nodes capable of providing effective forward progress are screened; based on cross-layer prediction indexes such as expected queuing delay, link expiration time and energy consumption rate, the comprehensive forwarding utility of the candidate nodes is calculated by using Mamdani fuzzy reasoning as the instant reward of Q learning; the next hop is selected in the filtered candidate set through Q learning, and multi-hop routing optimization oriented to reliability, low delay and low energy consumption is realized. The application reduces the exploration overhead caused by invalid candidate nodes in the high-density unmanned aerial vehicle network scene, and improves the foresight and stability of routing decision-making in a dynamic environment, which is superior to the existing unmanned aerial vehicle self-organizing network routing method.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) ad hoc network communication and intelligent routing technology, specifically to a cross-layer routing method for high-density flight ad hoc networks (FANETs), and in particular to an adaptive routing method based on geometric filtering and predictive fuzzy reward Q-learning (GPRF). Background Technology

[0002] With the development of unmanned aerial vehicle (UAV) technology, flying ad hoc networks (FANETs) composed of multiple UAVs have been widely used in emergency communication, disaster relief, and environmental monitoring. Due to the high-speed movement, frequent topology changes, and high network density of UAV nodes, routing design in FANETs faces significant challenges. Existing routing methods mainly include topology-based routing protocols, location-based routing methods, and adaptive routing mechanisms based on intelligent algorithms. However, topology-based routing methods struggle to maintain stable paths in highly dynamic environments; location-based methods, while reducing control overhead, typically rely solely on instantaneous location information, making it difficult to guarantee continuous link reliability; and reinforcement learning-based routing methods suffer from high state space dimensionality, slow convergence speed, and inaccurate reward design reflecting network state in complex dynamic environments. Furthermore, in high-density network scenarios, redundant forwarding and broadcast storms are further exacerbated, leading to network congestion, decreased data transmission efficiency, and increased energy consumption. Therefore, existing technologies struggle to simultaneously achieve routing stability, transmission efficiency, and adaptive capabilities in high-density, highly dynamic FANET environments. Summary of the Invention

[0003] To address the above issues, a cross-layer routing method (GPRF) for high-density flight ad hoc networks is proposed.

[0004] The technical solution of the present invention is as follows: a cross-layer routing method for high-density flight ad hoc networks, which consists of three parts: a neighbor perception module, a geometric filtering module, and a predictive fuzzy reward Q-learning routing decision module; The neighbor perception module acquires the local state of neighbors; the geometric filtering module performs geometric filtering oriented towards the destination node; the prediction fuzzy reward Q-learning routing decision module performs cross-layer prediction index calculation and fuzzy fusion, as well as hop-by-hop next-hop selection and Q-value update based on Q-learning.

[0005] The neighbor sensing module obtains local state information of a one-hop neighbor through periodic Hello message interactions, including location, velocity, queue length, and remaining energy. The location information is used for geometric filtering, the location and velocity are used to calculate the link duration (LET), the queue length is used to calculate the expected queuing delay (EQD), and the remaining energy is used to calculate the energy dissipation rate (EDR).

[0006] The geometric filtering module constructs a geometric filtering region oriented towards the destination node based on the relative direction between the current forwarding node and the destination node. It retains candidate neighbor nodes from the one-hop neighbors that meet the conditions of communication range, forward push and radial offset, so as to compress the candidate action space in high-density scenarios. When the filtered candidate set is empty, the original one-hop neighbor set is used through a backoff mechanism to ensure routing continuity.

[0007] The geometric filtering module constructs a candidate action set in the following manner: Let the current forwarding node be... The destination node is ,time The set of one-hop neighbors is Current forwarding node Candidate neighbor nodes and destination node The position vectors are respectively , and For any candidate neighbor node Its relative to the current forwarding node The relative position vector is: Establish forward axis and destination direction Align the local coordinate system and transform the relative position vector into the local coordinate system: in, Represents the three-dimensional coordinate transformation matrix acting on the relative position vector; Indicates candidate neighbor nodes Forward coordinates along the destination direction, and Indicates its two lateral coordinates relative to the forward axis, superscript A local coordinate system representing the direction of destination; radial offset of candidate neighbor nodes relative to the forward axis. Defined as: The maximum forward range of the geometric screening region for: Based on the communication range constraint, forward propagation constraint, and radial offset constraint, the geometric feasibility indicator of the candidate neighbor node is defined as follows: in, This is a binary geometric feasibility indicator. This indicates that the candidate neighbor node simultaneously satisfies the above three geometric conditions; Indicates the node communication radius. Describes the geometric filter radius and satisfies , Represent the Euclidean norm; obtain the filtered set of candidate neighbor nodes: When the filtered candidate neighbor set is empty, the original one-hop neighbor set is used as the backoff action set. The final set of valid candidate actions for hop-by-hop forwarding decision is defined as follows: in, Indicates the index of the current forwarding node. Indicates the index of the candidate neighbor node. Indicates the destination node. Indicates the current decision-making moment. This represents the transpose of a vector or matrix. Represents an empty set; This represents the set of candidate neighbor nodes obtained through geometric filtering. This represents the set of valid candidate actions after considering the fallback mechanism.

[0008] The predictive fuzzy reward Q-learning routing decision module calculates three cross-layer prediction indicators—LET, EQD, and EDR—for valid candidate neighbor nodes. After normalization, it obtains the comprehensive forwarding utility through the Mamdani fuzzy inference system. This comprehensive forwarding utility is used as the immediate reward for Q-learning. The routing strategy is iteratively optimized through hop-by-hop action selection and Q-value updates to maximize the expected long-term cumulative forwarding utility.

[0009] The hop-by-hop routing process is modeled as an optimization problem P1 with the objective of maximizing the expected long-term cumulative forwarding utility: in, Indicates the routing policy. Indicates in routing policy The following takes the expectation of the random routing process. Indicates the discount factor. Indicates the first Discount weighting at each decision moment Indicates the termination time of a hop-by-hop routing process; Indicates time The selected next-hop forwarding action, Indicates time Instant rewards Indicates from the current forwarding node Select candidate neighbor nodes The overall forwarding utility at that time; constraint C1 restricts the action to the set of valid candidate actions. Within; Constraint C2 indicates that when the geometric filter set is not empty, the selected next hop must satisfy the geometric feasibility condition; Constraint C3 uses the comprehensive forwarding utility of the selected next hop as an immediate reward.

[0010] For the set of valid candidate actions Candidate neighbor nodes The predictive fuzzy reward Q-learning routing decision module calculates the link duration (LET), expected queuing delay (EQD), and energy dissipation rate (EDR), respectively; where the current forwarding node... With candidate neighbor nodes The relative velocity vector is defined as: A constant velocity motion model is used within a short observation window. The link duration LET is defined as the maximum time offset by which the communication link remains within the communication radius in the future. Expected queuing delay (EQD) is defined as: The energy dissipation rate (EDR) is defined as: in, and These represent the current forwarding nodes. At any moment velocity vector and candidate neighbor nodes At any moment The velocity vector, This represents the relative velocity vector between the two. Indicates from the current moment The starting future time offset; Indicates candidate neighbor nodes At any moment The current queue length, This represents the average group service rate estimated within the recent observation window; and Representing candidate neighbor nodes respectively The remaining energy at two adjacent energy observation times, Indicates the energy observation interval; A larger value indicates a stronger predicted link persistence. and The smaller the value, the lower the degree of local congestion and the slower the energy consumption trend, respectively; Because LET, EQD, and EDR have different physical dimensions and numerical ranges, the three indices are normalized. The normalized three cross-layer prediction indices are expressed as follows: By inputting the above normalized indices into the Mamdani fuzzy inference system, and through fuzzification, rule-based inference, and centroid defuzzification, the comprehensive forwarding utility of candidate neighbor nodes is obtained: in, , , These represent the normalized values ​​of EQD, LET, and EDR, respectively. This represents the mapping formed by Mamdani fuzzy reasoning and defuzzification. Indicates candidate neighbor nodes The combined forwarding effect, and the instant reward .

[0011] The jump-by-hop decision algorithm is used to solve the optimization problem P1; at the decision time... Based on the local routing state of the current forwarding node As a state, with a set of valid candidate actions The candidate next-hop node in the table is used as the action, and according to... - Greedy strategy for choosing the next jump action: The first action selection method uses probability. Execution, the second random selection method uses probability. implement; Selecting candidate neighbor nodes After the next jump, let Perform packet forwarding and change the local routing state from Transfer to If the destination node has not yet been reached, update the Q value of the current state-action pair according to the following formula: in, Representing state Next, select candidate neighbor nodes. State-action value Indicates the learning rate. Indicates the probability of exploration. The discount factor is... Represents any candidate neighbor node in the set of valid candidate actions. Indicates that the packet is forwarded to the node. Then, at the next decision-making moment, the set of valid candidate actions is obtained based on the geometric screening and backoff mechanism. This indicates the next local routing state after the action is performed; when the packet reaches the destination node. When the time comes, the future reward item is set to 0; by repeatedly performing candidate screening, predictive utility evaluation, action selection and Q-value update, a hop-by-hop forwarding strategy for high-density FANET is obtained.

[0012] The beneficial effects of this invention are as follows: This method reduces the number of candidate next-hop nodes through a destination direction geometric filtering mechanism, thereby reducing the ineffective exploration overhead of reinforcement learning routing decisions in high-density scenarios. By introducing cross-layer prediction indicators such as link expiration time, expected queuing delay, and energy consumption rate, it improves the routing decision's ability to perceive link stability, node congestion status, and energy change trends. By using the comprehensive forwarding utility obtained from fuzzy inference as the immediate reward for Q-learning, it enhances the foresight and adaptability of next-hop selection. Compared with existing routing methods for UAV ad hoc networks, this invention can improve packet delivery rate, reduce average end-to-end latency and energy consumption, thereby improving the reliability, energy efficiency, and scalability of high-density flight ad hoc networks. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the set of one-hop neighbor states of the current forwarding node.

[0014] Figure 2 This is a schematic diagram of the candidate neighbor node set after geometric filtering based on the destination direction.

[0015] Figure 3 This is a schematic diagram of next-hop selection and state transition on the filtered candidate set.

[0016] Figure 4 This is a schematic diagram of a high-density flight self-organizing network system model.

[0017] Figure 5 This is a schematic diagram comparing the average end-to-end latency of GPRF and the baseline algorithm in a scenario with 50 drone nodes.

[0018] Figure 6 This is a schematic diagram comparing the average end-to-end latency of GPRF and the baseline algorithm in a scenario with 100 drone nodes.

[0019] Figure 7 This is a schematic diagram comparing the average end-to-end latency of GPRF and the baseline algorithm in a scenario with 150 drone nodes.

[0020] Figure 8 This is a diagram showing the energy consumption comparison between GPRF and the baseline algorithm in a scenario with 50 drone nodes.

[0021] Figure 9 This is a diagram comparing the energy consumption of GPRF and the baseline algorithm in a scenario with 100 drone nodes.

[0022] Figure 10 This is a diagram showing the energy consumption comparison between GPRF and the baseline algorithm in a scenario with 150 drone nodes.

[0023] Figure 11 This is a diagram comparing the packet delivery rates of GPRF and the baseline algorithm in a scenario with 50 drone nodes.

[0024] Figure 12 This is a diagram comparing the packet delivery rates of GPRF and the baseline algorithm in a scenario with 100 drone nodes.

[0025] Figure 13 This is a diagram comparing the packet delivery rates of GPRF and the baseline algorithm in a scenario with 150 drone nodes. Detailed Implementation

[0026] The specific embodiments of the present invention will be described in detail below.

[0027] This method achieves route optimization by constructing a multi-module collaborative mechanism: First, a geometric filtering module is used to spatially constrain candidate forwarding nodes, reducing invalid forwarding nodes and thus lowering network redundancy and broadcast overhead. Second, a predictive fuzzy reward Q-learning routing decision module calculates three cross-layer prediction metrics for candidate neighbor nodes: Link Duration (LET), Expected Queuing Delay (EQD), and Energy Dissipation Rate (EDR), to evaluate the link stability, local congestion level, and energy consumption trend of candidate nodes. Third, the above cross-layer prediction metrics are normalized and input into the Mamdani fuzzy inference system. Through fuzzification, rule-based reasoning, and centroid defuzzification, the comprehensive forwarding utility of candidate neighbor nodes is obtained, and this comprehensive forwarding utility is used as the immediate reward for Q-learning. Finally, the next-hop routing strategy is iteratively optimized through hop-by-hop action selection and Q-value updates in Q-learning. This method can effectively improve the data transmission success rate in high-density FANETs, ​​reduce latency and energy consumption, and enhance the network's robustness and adaptability in dynamic environments.

[0028] This invention specifically proposes a cross-layer routing method for high-density flight ad hoc networks, with the overall system framework as follows: Figure 1 As shown. The main idea is: in a high-density UAV network, firstly, a destination-direction geometric filtering mechanism is used to filter out neighboring nodes that are not conducive to forward forwarding, reducing the number of candidate next-hop nodes and ineffective explorations during reinforcement learning; then, by combining cross-layer prediction information such as link stability, node congestion status, and energy consumption trends, the comprehensive forwarding utility of candidate neighboring nodes is constructed, and this is used as an immediate reward for the Q-learning algorithm, thereby improving the reliability, latency performance, and energy utilization efficiency of multi-hop data forwarding in the ad hoc flight network.

[0029] This invention proposes a cross-layer routing method for high-density flight ad hoc networks. The overall system framework is as follows: Figure 1 As shown. The overall system framework includes a neighbor perception module, a geometric filtering module, and a predictive fuzzy reward Q-learning routing decision module. The system periodically perceives the location, speed, queue status, and remaining energy of one-hop neighbors through UAV nodes, constructs a neighbor state table for the current forwarding node in real time, and completes candidate neighbor node selection, forwarding reward calculation, and next-hop routing decision based on this table, thereby realizing intelligent distributed routing in high-density flight ad hoc networks.

[0030] First, this invention analyzes the problems of existing methods. In high-density flight ad hoc network scenarios, as the number of UAV nodes increases, the number of one-hop neighbors of the current forwarding node increases significantly, leading to a rapid expansion of the routing candidate action space. Existing reinforcement learning routing methods are prone to problems such as high exploration overhead, slow convergence speed, and increased invalid forwarding. On the other hand, existing routing methods based on geometric location or rule-driven approaches typically select the next hop mainly based on instantaneous distance, direction, or forward progress, lacking comprehensive predictive capabilities for link stability, node congestion status, and energy change trends, making it difficult to guarantee reliability, latency performance, and energy efficiency in highly dynamic networks. To address the above problems, this invention combines... Figure 2 , Figure 3 and Figure 4 The process shown first obtains the candidate neighbor node state set from the original one-hop neighbor set of the current forwarding node, such as... Figure 2 As shown; then, based on the spatial directional relationship between the current forwarding node and the destination node, a destination direction geometric filtering region is constructed to filter the original neighbor set, resulting in a candidate neighbor node set with effective forward progress, as shown. Figure 3 As shown; finally, based on the filtered candidate neighbor node set, and combining cross-layer prediction indicators such as link duration, expected queuing delay, and energy consumption rate, a comprehensive forwarding reward is generated through fuzzy inference, and this reward is used as the immediate reward for Q-learning for next-hop selection and Q-value update, as follows. Figure 4 As shown, this improves routing reliability and reduces latency and energy consumption in high-density flight ad hoc networks.

[0031] (1) Neighbor perception module.

[0032] This invention models a high-density flight ad hoc network as a time-varying three-dimensional UAV ad hoc network. Let the set of UAV nodes be... The ground control station is Then the 3D UAV self-organizing network at time This can be represented as a time-varying directed graph: (1) in, , Indicates time The set of valid wireless links in a 3D UAV ad hoc network. For any UAV node Its three-dimensional position and velocity are expressed as follows: (2) (3) Each drone node periodically broadcasts neighbor awareness information including node identifier, location, speed, queue length, average service rate, and remaining energy. (Current forwarding node) Maintain a set of one-hop neighbors based on the most recently received neighbor perception information. And the neighbor state table. For any neighbor node The current forwarding node can obtain its location. ,speed Queue length Average service speed and remaining energy The above information is used as input for candidate neighbor node selection and routing decisions.

[0033] (2) Candidate neighbor node geometric filtering module.

[0034] Before making the next-hop routing decision, this invention first performs destination direction geometric filtering on the one-hop neighbor set of the current forwarding node to reduce the candidate action space in high-density scenarios. Let the current forwarding node be... The destination node is The positions of the two are as follows: (4) (5) For any neighbor node Calculate its relative position vector with respect to the current forwarding node: (6) Right now: (7) Then, construct the rotation matrix. This aligns the axial direction of the local coordinate system with the direction from the current forwarding node to the destination node. Alignment. Neighbor nodes The coordinates in the local coordinate system in the destination direction are represented as follows: (8) in, This indicates the forward progress of neighboring nodes along the destination direction. and This represents the lateral offset of a neighboring node relative to the destination direction axis. Further, the radial offset of a neighboring node is defined as: (9) This invention sets the target direction geometric filtering region as a local cylindrical region extending along the target direction, with a height of: (10) in, The radius of the UAV node communication; the radius of the cylindrical filtering area is... And satisfy If neighboring nodes It simultaneously satisfies the communication range constraint, forward progress constraint, and radial offset constraint, that is: (11) Then the neighboring node is considered to meet the geometric feasibility condition. Therefore, the geometric feasibility indicator variable is defined as follows: (12) The set of candidate neighbor nodes after geometric filtering is defined as follows: (13) Considering that the geometric filtering result may be empty in some dynamic scenarios, in order to avoid routing interruption, this invention sets an empty set fallback mechanism, defining the final effective candidate set used for routing decision as: (14) in, Indicates the current forwarding node at time [time]. The set of valid candidate actions.

[0035] (3) Predicting fuzzy reward Q-learning routing decision module.

[0036] After obtaining the valid candidate set Subsequently, this invention performs cross-layer prediction and evaluation on each candidate neighbor node. Evaluation metrics include link duration, expected queuing delay, and energy consumption rate, which are used to characterize link stability, node congestion state, and energy change trend, respectively. Specifically, this includes: 1) Link duration.

[0037] Current forwarding node With candidate neighbor nodes The relative velocity between them is expressed as: (14) Within a short observation window, a uniform motion approximation model is adopted, assuming... Indicates from the current moment The link duration is defined as follows: (The initial future time offset is used to define the link duration.) (14) in, The larger the value, the longer the link between the current forwarding node and the candidate neighbor node is expected to remain active.

[0038] 2) Expected queuing delay.

[0039] Set candidate neighbor nodes The current queue length is Its average service rate within the most recent observation window is The expected queuing delay of a newly arriving packet at a candidate neighbor node is defined as follows: in, The smaller the value, the lower the current congestion level of the candidate neighbor node, and the less queuing and waiting may occur when data packets pass through that node.

[0040] 3) Energy consumption rate.

[0041] Set candidate neighbor nodes The remaining energy at two adjacent observation times are respectively and ,in Let the energy observation interval be denoted as , then the energy consumption rate is defined as: in, The smaller the value, the slower the energy consumption trend of the candidate neighbor node, and the better its energy sustainability when it is the next hop node.

[0042] (4) Predicting fuzzy reward generation module.

[0043] Since link duration, expected queuing delay, and energy consumption rate have different physical meanings and numerical ranges, this invention first normalizes the above three indicators to obtain... , and For candidate neighbor nodes Construct its predictive cross-layer feature vector: (14) Subsequently, the three normalized indicators are input into the Mamdani-type fuzzy inference system. Expected queuing delay and energy consumption rate are mapped to low, medium, and high fuzzy linguistic variables, respectively, while link duration is mapped to short, medium, and long fuzzy linguistic variables. This invention constructs a fuzzy rule base based on the principle that "the lower the expected queuing delay, the longer the link duration, and the lower the energy consumption rate, the more suitable the candidate neighbor node is as the next hop." The comprehensive forwarding utility of the candidate neighbor node is obtained through fuzzy inference and defuzzification. (15) in, This represents the process of fuzzy reasoning and defuzzification. Indicates candidate neighbor nodes Relative to the current forwarding node The overall forwarding effect.

[0044] (5) Next-hop routing decision module based on Q-learning.

[0045] This invention models the hop-by-hop routing process as a sequential decision-making process based on Q-learning. At each routing decision point, the current forwarding node selects the next-hop node based on the effective candidate set and the predicted fuzzy reward, and updates the Q-value after packet forwarding. The specific definitions are as follows: 1) State and action.

[0046] Represent the local routing state of the current forwarding node as follows: , will be the effective candidate set The candidate neighbor nodes are used as the action set. When selecting a candidate neighbor node... When it becomes the next jump, the corresponding action is represented as: (15) 2) Rewards.

[0047] When selecting candidate neighbor nodes As the next hop, this invention uses the comprehensive forwarding utility obtained by the predictive fuzzy reward generation module as the immediate reward: 3) Action selection and Q-value update.

[0048] This invention adopts The -greedy strategy selects the next hop node: (16) in, Indicates the state Next, select candidate neighbor nodes. As the Q value for the next hop, To explore probabilities, the current forwarding node forwards data packets to the selected next-hop node. Then, the routing state transitions to If the data packet has not yet reached the destination node or ground control station, the Q value is updated according to the following formula: (17) in, For learning rate, As a discount factor, Indicates the next hop node The effective candidate set at the next decision moment. If the data packet reaches the destination node or ground control station, the future payoff term is set to zero, and the Q value is updated as follows: (18) By repeatedly executing the processes of neighbor state awareness, destination direction geometric filtering, cross-layer prediction index calculation, fuzzy reward generation, next-hop selection, and Q-value update, this invention can gradually learn a hop-by-hop routing strategy suitable for high-density flight ad hoc networks.

[0049] (6) Routing optimization objectives.

[0050] In summary, this invention models the hop-by-hop routing process in high-density ad hoc flying networks as a constrained sequential decision problem with the objective of maximizing the expected long-term cumulative forwarding utility: in, Indicates the routing policy. Indicates in routing policy The following takes the expectation of the random routing process. Indicates the discount factor. Indicates the first Discount weighting at each decision moment Indicates the termination time of a hop-by-hop routing process; Indicates time The selected next-hop forwarding action, Indicates time Instant rewards Indicates from the current forwarding node Select candidate neighbor nodes The overall forwarding utility at that time; constraint C1 restricts the action to the set of valid candidate actions. Within; Constraint C2 indicates that when the geometric filter set is not empty, the selected next hop must satisfy the geometric feasibility condition; Constraint C3 uses the comprehensive forwarding utility of the selected next hop as an immediate reward.

[0051] It should be noted that the following embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit the scope of protection of the present invention. In this embodiment, the simulation platform uses NS2, the routing algorithm is implemented in Python, and the simulation scenario is a three-dimensional low-altitude flight ad hoc network scenario. The network consists of multiple UAV nodes and a ground control station. UAV nodes forward data to each other via wireless links, and some UAV nodes can communicate directly with the ground control station when they approach it. The GPRF method in this embodiment mainly includes a neighbor perception module, a geometric filtering module, and a predictive fuzzy reward Q-learning routing decision module.

[0052] Step 1: Implement the content of each module.

[0053] The first step is to implement the neighbor awareness module. Each UAV node in the system periodically broadcasts lightweight neighbor awareness information, including node identifier, current location, movement speed, queue length, average service rate, and remaining energy. The current forwarding node maintains a one-hop neighbor set and a neighbor state table based on the most recently received neighbor awareness information. This neighbor state table serves as the basis for subsequent candidate neighbor node selection, cross-layer index calculation, and next-hop selection. This invention does not require full-network topology flooding, relying only on one-hop neighbor state awareness, making it more suitable for highly dynamic flight ad hoc network scenarios. This framework relies on one-hop Hello messages to maintain the neighbor table and uses neighbor positions, speeds, queue lengths, and remaining energy as the basis for routing decisions.

[0054] The second step is to implement the geometric filtering module. Before selecting the next hop, the current forwarding node first obtains the location information of the destination node and constructs a local geometric filtering region based on the direction from the current forwarding node to the destination node. The system first obtains the original set of one-hop neighbors within the communication range of the current forwarding node, such as... Figure 2 As shown; then, neighbor nodes that cannot provide effective forward progress, deviate significantly from the destination direction, or do not meet the communication range requirements are removed, resulting in a filtered set of candidate neighbor nodes, such as... Figure 3 As shown. If the candidate set is empty after filtering, the original one-hop neighbor set is used as a fallback to avoid routing interruption due to overly strict filtering. This geometric filtering module can compress the candidate action space before entering the reinforcement learning decision, reducing invalid exploration in high-density scenarios. GPRF introduces destination direction geometric filtering before reinforcement learning decision to compress the candidate action space in high-density FANETs.

[0055] The third step involves predicting the fuzzy reward Q-learning routing decision module. For the valid candidate neighbor nodes obtained after geometric filtering, the system calculates the link duration, expected queuing delay, and energy consumption rate. The link duration is used to predict how long the wireless link between the current forwarding node and the candidate neighbor node can be maintained; the expected queuing delay reflects whether the candidate neighbor node is currently congested; and the energy consumption rate reflects the energy change trend of the candidate neighbor node. Through these three indicators, the system can evaluate candidate neighbor nodes from three aspects: link stability, node congestion status, and energy sustainability, rather than making forwarding decisions solely based on distance or direction.

[0056] Because link duration, expected queuing delay, and energy consumption rate have different physical meanings, the system first normalizes these three indicators before inputting them into the Mamdani fuzzy inference system. Expected queuing delay and energy consumption rate are categorized into low, medium, and high fuzzy levels, while link duration is categorized into short, medium, and long fuzzy levels. The system constructs a fuzzy rule base based on the principle that "the lower the queuing delay, the longer the link duration, and the lower the energy consumption rate, the more suitable the candidate neighbor node is as the next hop," and obtains the comprehensive forwarding utility of the candidate neighbor nodes through fuzzy inference and defuzzification. This comprehensive forwarding utility serves as the immediate reward for Q-learning and is used for subsequent next hop selection, such as... Figure 4 As shown.

[0057] At each routing decision point, the current forwarding node considers neighbor nodes from the set of valid candidate neighbor nodes as optional actions, uses the current local routing state as the state input, and uses the predicted fuzzy reward as the immediate reward. The system employs an ε-greedy strategy to select the next hop among candidate neighbor nodes: in most cases, it selects the candidate neighbor node with the higher current Q value; in a few cases, it randomly selects candidate neighbor nodes to maintain exploration capability. After the current forwarding node completes the forwarding of data packets, it updates the Q value based on whether the packet has reached the destination node or the ground control station. If the packet has reached the destination node, the future reward is set to zero; if it has not yet reached the destination node, the candidate neighbor node selection, fuzzy reward calculation, and Q value update process are re-executed at the new current forwarding node.

[0058] Step 2: Run the GPRF hop-by-hop routing decision algorithm.

[0059] During system initialization, the communication radius, geometric filtering radius, learning rate, discount factor, exploration probability, initial energy, node motion model, service flow parameters, and initial values ​​of the Q-table are set. For each data packet to be forwarded, the current forwarding node first reads its own neighbor state table to obtain the currently communicable one-hop neighbor nodes; then it performs destination direction geometric filtering to obtain a valid candidate set; next, it calculates the link duration, expected queuing delay, and energy consumption rate for each candidate neighbor node, and uses a fuzzy inference system to obtain the corresponding comprehensive forwarding utility; then, it selects the next-hop node according to the ε-greedy policy and forwards the data packet to that node; finally, it updates the Q-table based on the forwarding results. The above process is repeated for each hop-by-hop forwarding until the data packet reaches the ground control station or the routing termination condition is met.

[0060] Step 3: Perform simulation to verify the algorithm performance.

[0061] To verify the effectiveness of the method of the present invention, this embodiment constructs a three-dimensional flight ad hoc network simulation scenario. The simulation area is set as follows: The number of drone nodes was set to 50, 100, and 150 to represent different network density scenarios; the drone speed was set to 10–40 m / s, and a three-dimensional Gauss-Markov mobile model was used to simulate topology changes in low-altitude flight environments; the communication radius was set to 250 m, the initial node energy was set to 1000 J, the communication standard adopted was IEEE 802.11n, the service model adopted was UDP-based CBR service, the data rate was set to 2 Mbps, and the carrier frequency was set to 2.4 GHz.

[0062] This embodiment selects QRF, OLSR+, and QFAN as comparison algorithms, and denotes the method proposed in this invention as GPRF. All algorithms are run under the same network configuration, mobility model, and service model, with each experiment running 100 rounds of performance evolution. Evaluation metrics include average end-to-end latency, network energy consumption, and packet delivery rate, used to evaluate the real-time performance, energy efficiency, and reliability of the routing method, respectively.

[0063] First, the average end-to-end latency under different network densities is verified. Figure 5 , Figure 6 and Figure 7 The average end-to-end latency variations for different routing algorithms are presented in scenarios with 50, 100, and 150 drone nodes, respectively. Figure 5 It can be seen that, in a 50-node scenario, the GPRF method proposed in this invention maintains the lowest average end-to-end latency overall; Figure 6 It can be seen that when the number of nodes increases to 100, GPRF can still maintain a low and relatively stable latency level during performance evolution; from Figure 7 It can be seen that in a high-density scenario with 150 nodes, the latency fluctuations and latency values ​​of other algorithms increase significantly, while GPRF still achieves a lower average end-to-end latency. These results demonstrate that this invention reduces invalid candidate neighbor nodes through destination-direction geometric filtering and avoids selecting nodes with severe congestion or short link durations by predicting fuzzy rewards, thereby reducing the waiting time and path reconstruction overhead of data packets during multi-hop forwarding.

[0064] Secondly, the energy consumption under different network densities was verified. Figure 8 , Figure 9 and Figure 10 The energy consumption variations of different routing algorithms are presented for scenarios with 50, 100, and 150 drone nodes, respectively. Figure 8It can be seen that in the 50-node scenario, the energy consumption growth rate of GPRF is lower than that of most of the comparison algorithms; Figure 9 It can be seen that in the 100-node scenario, as the performance evolution rounds increase, GPRF consistently maintains a low cumulative energy consumption; from Figure 10 It is evident that GPRF's energy consumption advantage is more pronounced in high-density scenarios with 150 nodes. This result demonstrates that, with increased node density and a larger number of candidate neighbors, this invention can reduce redundant exploration and invalid forwarding through geometric filtering, and avoid overuse of candidate neighbor nodes with unfavorable energy states through energy consumption rate metrics, thereby reducing overall network energy consumption.

[0065] Finally, the packet delivery rate under different network densities was verified. Figure 11 , Figure 12 and Figure 13 The changes in packet delivery rate for different routing algorithms are presented in scenarios with 50, 100, and 150 drone nodes, respectively. Figure 11 It can be seen that in a 50-node scenario, GPRF can maintain a high packet delivery rate; from Figure 12 It can be seen that, in a 100-node scenario, GPRF's packet delivery rate is generally higher than that of QRF, OLSR+, and QFAN; Figure 13 It can be seen that even in a high-density scenario with 150 nodes, GPRF can still maintain a high and relatively stable packet delivery rate. This result indicates that by combining link duration, expected queuing delay, and energy consumption rate to comprehensively evaluate candidate neighbor nodes, this invention can reduce the probability of selecting unstable links, congested nodes, or energy-inefficient nodes, thereby improving the ability of data packets to successfully reach the destination node or ground control station.

[0066] comprehensive Figures 5 to 13 As can be seen, under different numbers of UAV nodes and different network densities, the GPRF method proposed in this invention exhibits good overall performance in terms of average end-to-end latency, energy consumption, and packet delivery rate. With the increase in the number of UAV nodes, the candidate action space expands, link changes become more frequent, and node congestion becomes more pronounced in high-density scenarios. This invention, through a continuous processing flow of destination direction geometric filtering, cross-layer prediction index calculation, prediction fuzzy reward generation, and Q-learning routing decision, can effectively compress the candidate action space and improve the next-hop selection's ability to perceive link stability, node congestion status, and energy consumption trends, thereby achieving more reliable, low-latency, and energy-aware data forwarding.

Claims

1. A cross-layer routing method for high-density flight ad hoc networks, characterized in that, It consists of three parts: a neighbor perception module, a geometric filtering module, and a Q-learning routing decision module for predicting fuzzy rewards. The neighbor perception module acquires the local state of neighbors; the geometric filtering module performs geometric filtering oriented towards the destination node; the prediction fuzzy reward Q-learning routing decision module performs cross-layer prediction index calculation and fuzzy fusion, as well as hop-by-hop next-hop selection and Q-value update based on Q-learning.

2. The cross-layer routing method for high-density flight ad hoc networks according to claim 1, characterized in that, The neighbor sensing module obtains local state information of a one-hop neighbor through periodic Hello message interactions, including location, velocity, queue length, and remaining energy. The location information is used for geometric filtering, the location and velocity are used to calculate the link duration (LET), the queue length is used to calculate the expected queuing delay (EQD), and the remaining energy is used to calculate the energy dissipation rate (EDR).

3. The cross-layer routing method for high-density flight ad hoc networks according to claim 1, characterized in that, The geometric filtering module constructs a geometric filtering region oriented towards the destination node based on the relative direction between the current forwarding node and the destination node. It retains candidate neighbor nodes from the one-hop neighbors that meet the conditions of communication range, forward push and radial offset, so as to compress the candidate action space in high-density scenarios. When the filtered candidate set is empty, the original one-hop neighbor set is used through a backoff mechanism to ensure routing continuity.

4. The cross-layer routing method for high-density flight ad hoc networks according to claim 3, characterized in that, The geometric filtering module constructs a candidate action set in the following manner: Let the current forwarding node be... The destination node is ,time The set of one-hop neighbors is Current forwarding node Candidate neighbor nodes and destination node The position vectors are respectively , and For any candidate neighbor node Its relative to the current forwarding node The relative position vector is: Establish forward axis and destination direction Align the local coordinate system and transform the relative position vector into the local coordinate system: in, Represents the three-dimensional coordinate transformation matrix acting on the relative position vector; Indicates candidate neighbor nodes Forward coordinates along the destination direction, and Indicates its two lateral coordinates relative to the forward axis, superscript A local coordinate system representing the direction of destination; radial offset of candidate neighbor nodes relative to the forward axis. Defined as: The maximum forward range of the geometric screening region for: Based on the communication range constraint, forward propagation constraint, and radial offset constraint, the geometric feasibility indicator of the candidate neighbor node is defined as follows: in, This is a binary geometric feasibility indicator. This indicates that the candidate neighbor node simultaneously satisfies the above three geometric conditions; Indicates the node communication radius. Describes the geometric filter radius and satisfies , Represent the Euclidean norm; obtain the filtered set of candidate neighbor nodes: When the filtered candidate neighbor set is empty, the original one-hop neighbor set is used as the backoff action set. The final set of valid candidate actions for hop-by-hop forwarding decision is defined as follows: in, Indicates the index of the current forwarding node. Indicates the index of the candidate neighbor node. Indicates the destination node. Indicates the current decision-making moment. This represents the transpose of a vector or matrix. Represents an empty set; This represents the set of candidate neighbor nodes obtained through geometric filtering. This represents the set of valid candidate actions after considering the fallback mechanism.

5. The cross-layer routing method for high-density flight ad hoc networks according to claim 1, characterized in that, The predictive fuzzy reward Q-learning routing decision module calculates three cross-layer prediction indicators—LET, EQD, and EDR—for valid candidate neighbor nodes. After normalization, these indicators are used to obtain the comprehensive forwarding utility through the Mamdani fuzzy inference system. This comprehensive forwarding utility is then used as the immediate reward for Q-learning. The routing strategy is iteratively optimized through hop-by-hop action selection and Q-value update learning to maximize the expected long-term cumulative forwarding utility.

6. The cross-layer routing method for high-density flight ad hoc networks according to claim 5, characterized in that, The hop-by-hop routing process is modeled as an optimization problem P1 with the objective of maximizing the expected long-term cumulative forwarding utility: in, Indicates the routing policy. Indicating in routing policy The following takes the expectation of the random routing process. Indicates the discount factor. Indicates the first Discount weighting at each decision moment Indicates the termination time of a hop-by-hop routing process; Indicates time The selected next-hop forwarding action, Indicates time Instant rewards Indicates from the current forwarding node Select candidate neighbor nodes The overall forwarding utility at that time; constraint C1 restricts the action to the set of valid candidate actions. Within; Constraint C2 indicates that when the geometric filter set is not empty, the selected next hop must satisfy the geometric feasibility condition; Constraint C3 uses the comprehensive forwarding utility of the selected next hop as an immediate reward.

7. The cross-layer routing method for high-density flight ad hoc networks according to claim 6, characterized in that, For the set of valid candidate actions Candidate neighbor nodes The predictive fuzzy reward Q-learning routing decision module calculates the link duration (LET), expected queuing delay (EQD), and energy dissipation rate (EDR), respectively; where the current forwarding node... With candidate neighbor nodes The relative velocity vector is defined as: A constant velocity motion model is used within a short observation window. The link duration LET is defined as the maximum time offset by which the communication link remains within the communication radius in the future. Expected queuing delay (EQD) is defined as: The energy dissipation rate (EDR) is defined as: in, and These represent the current forwarding nodes. At any moment velocity vector and candidate neighbor nodes At any moment The velocity vector, This represents the relative velocity vector between the two. Indicates from the current moment The starting future time offset; Indicates candidate neighbor nodes At any moment The current queue length, This represents the average group service rate estimated within the recent observation window; and Representing candidate neighbor nodes respectively The remaining energy at two adjacent energy observation times, Indicates the energy observation interval; A larger value indicates a stronger predicted link persistence. and The smaller the value, the lower the degree of local congestion and the slower the energy consumption trend, respectively; Because LET, EQD, and EDR have different physical dimensions and numerical ranges, the three indices are normalized. The normalized three cross-layer prediction indices are expressed as follows: By inputting the above normalized indices into the Mamdani fuzzy inference system, and through fuzzification, rule-based inference, and centroid defuzzification, the comprehensive forwarding utility of candidate neighbor nodes is obtained: in, , , These represent the normalized values ​​of EQD, LET, and EDR, respectively. This represents the mapping formed by Mamdani fuzzy reasoning and defuzzification. Indicates candidate neighbor nodes The combined forwarding effect, and the instant reward .

8. The cross-layer routing method for high-density flight ad hoc networks according to claim 7, characterized in that, The jump-by-hop decision algorithm is used to solve the optimization problem P1; at the decision time... Based on the local routing state of the current forwarding node As a state, with a set of valid candidate actions The candidate next-hop node in the table is used as the action, and according to... - Greedy strategy for choosing the next jump action: The first action selection method uses probability. Execution, the second random selection method uses probability. implement; Selecting candidate neighbor nodes After the next jump, let Perform packet forwarding and change the local routing state from Transfer to If the destination node has not yet been reached, update the Q value of the current state-action pair according to the following formula: in, Representing state Next, select candidate neighbor nodes. State-action value Indicates the learning rate. Indicates the probability of exploration. The discount factor is... Represents any candidate neighbor node in the set of valid candidate actions. Indicates that the packet is forwarded to the node. Then, at the next decision-making moment, the set of valid candidate actions is obtained based on the geometric screening and backoff mechanism. This indicates the next local routing state after the action is performed; when the packet reaches the destination node. When the time comes, the future reward item is set to 0; by repeatedly performing candidate screening, predictive utility evaluation, action selection and Q-value update, a hop-by-hop forwarding strategy for high-density FANET is obtained.

Images