A self-evolving routing method for UAV networks based on enhanced twins

Abstract

This invention discloses a self-evolving routing method for Unmanned Aerial Vehicle (UAV) networks based on enhanced twins. Applied to the UAV network field, it addresses the problem that existing routing mechanisms struggle to simultaneously meet the requirements of flexibility and low overhead in the dynamic and resource-constrained UAV networks. Specifically, this invention designs a self-evolving routing technology for UAV networks based on enhanced twins. This technology mitigates the deviations in routing decisions caused by synchronization errors between the physical space and the enhanced twin space of the UAV network. Furthermore, it dynamically and intelligently optimizes the routing pool according to the network environment, achieving intelligent adaptive capability of the routing mechanism to the external environment, thus ensuring efficient data transmission and network stability.

Description

Technical Field

[0001] This invention belongs to the field of unmanned aerial vehicle (UAV) networks, and specifically relates to a UAV network routing technology. Background Technology

[0002] With the rapid development of electronic technology, sensors, wireless communication technology, big data, and cloud computing, the hardware and software performance of Unmanned Aerial Vehicles (UAVs) has been greatly improved, and they are widely used in various fields, such as collaborative reconnaissance, target tracking, search and rescue, and agricultural spraying. When dealing with complex tasks, individual UAVs often face limitations in coverage and processing power, making it difficult to fully meet the diversity and complexity of application tasks. To effectively address this challenge, multiple UAVs often form a network to work collaboratively. Through collaborative work, UAVs can share information, effectively distribute workloads, and coordinate resource utilization. This networked approach not only improves the adaptability and flexibility of the overall system but also more effectively responds to changing needs in different scenarios, achieving efficient task execution.

[0003] Therefore, UAV Ad Hoc Networks (UAVNETs) have become an important direction in UAV development, bringing more efficient, flexible, and reliable solutions to various applications. Through UAVNETs, ​​efficient data transmission links are established between UAVs, enabling rapid information exchange and sharing. This network structure endows UAV swarms with greater intelligence and adaptability, allowing them to efficiently handle complex mission scenarios. For example, in search and rescue missions, UAVs can autonomously network to achieve comprehensive searches within a region and promptly transmit target information to the command center, thereby enabling rapid response and coordinated action, improving rescue efficiency and minimizing potential risks. Furthermore, because UAVs can communicate and collaborate with each other, even if some UAVs malfunction or are interfered with, the remaining UAVs can still work together to complete the mission, ensuring its successful execution.

[0004] The flexible deployment and rapid response capabilities of UAV ad hoc networks enable them to collaboratively complete various complex tasks more efficiently, and their application scenarios are constantly expanding. However, the high-speed mobility of UAVs in the air also brings frequent changes and reorganizations in network topology, making the communication environment, network topology, and application requirements faced by UAV ad hoc networks increasingly complex and diverse. This poses a serious challenge to the dynamism and flexibility of routing algorithms. Traditional routing algorithms have simple decision-making mechanisms and low flexibility, often failing to adapt to rapidly changing network conditions, potentially leading to problems such as packet loss, increased latency, or network congestion. In the face of complex and ever-changing network states, routing selection can be dynamically adjusted based on various factors, including network topology information, geographical location information, signal strength, and transmission quality. Furthermore, predictive models, bio-inspired algorithms, and machine learning techniques can be combined to adapt to changes in the network environment. However, these intelligent routing algorithms typically require significant computational and communication resources, making them difficult to apply effectively in resource-constrained UAV networks.

[0005] Therefore, in resource-constrained UAV networks, introducing augmented twin technology, which enhances the physical network's perception, communication, and computing capabilities, is crucial for improving routing mechanisms. Furthermore, due to the high-speed movement of UAVs and the instability of the wireless environment, synchronization errors exist between the physical space and the augmented twin space of the UAV network. This error can potentially bias routing decisions in the physical space. Therefore, minimizing the impact of synchronization errors on routing selection is essential. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention proposes a self-evolving routing method for UAV networks based on enhanced twins. This method comprehensively considers various factors within the network, corrects for the deviations in routing decisions caused by synchronization errors between the UAV network's physical space and the enhanced twin space, and ultimately selects the routing algorithm best suited to environmental changes. Furthermore, it dynamically and intelligently optimizes the routing pool based on the network environment, achieving intelligent adaptive capabilities of the routing mechanism to the external environment, ensuring efficient data transmission and network stability.

[0007] The technical solution adopted in this invention is as follows: a self-evolving routing method for UAV networks based on enhanced twins. The routing system on which this method is based includes: a drone, a server, and a virtual space running on the server. The virtual space includes several enhanced twins, each of which corresponds to a drone. The method specifically includes the following steps:

[0008] S1. The drone transmits network environment data to its corresponding enhanced twin;

[0009] S2. Enhanced twin performs routing simulation based on received network environment data to obtain the corresponding indicator data for different routing algorithms in the active routing pool.

[0010] S3. Based on the metric data corresponding to different routing algorithms in the active routing pool;

[0011] S4. Adjust the data of each indicator according to the error evaluation model to obtain the scores of different routing algorithms in the active routing pool;

[0012] S5. Select the optimal route for each drone based on the scores of different routing algorithms in the active routing pool.

[0013] Furthermore, it also includes optimizing the routing algorithm in the active routing pool using a connectivity-based federated DQN (Deep Q-Network) algorithm.

[0014] The beneficial effects of this invention are as follows: This invention designs a novel intelligent routing mechanism using enhanced twin technology. Specifically, addressing the synchronization error between the physical space and the enhanced twin space, an error-aware intelligent routing selection technology is designed. This technology objectively and impartially evaluates routing performance through an error correction model, achieving the ability to make the optimal selection among multiple available routes. This invention also designs a connectivity-based federated DQN technology for intelligently optimizing the active route pool in dynamic network environments. By evaluating the performance of active routes in the twin space and considering factors such as network node distribution, link status, topology changes, and service load, a routing algorithm selection strategy that dynamically learns and adapts to changes in the network environment using DQN is adopted. Furthermore, considering the differences in data referenceability among network nodes, a federated learning method is used to jointly train and aggregate the DQN models of each node, thereby improving the matching and adaptability of the routing algorithm to the network environment and achieving intelligent optimization of the active route pool. Attached Figure Description

[0015] Figure 1 This is a diagram of an intelligent routing architecture based on enhanced twins;

[0016] Figure 2 This is a flowchart of error-aware intelligent routing selection based on enhanced twins;

[0017] Figure 3 It is a routing algorithm graph in the routing algorithm library;

[0018] Figure 4 This is a flowchart of a federated DQN active routing pool intelligent optimization system based on connectivity. Detailed Implementation

[0019] To facilitate understanding of the technical content of this invention by those skilled in the art, the following description, in conjunction with the accompanying drawings, further illustrates the invention.

[0020] As the application scenarios of UAV ad hoc networks continue to expand, the communication environment, network topology, and application requirements they face are becoming increasingly complex and diverse. This poses a significant challenge to the routing mechanism of UAV networks: traditional routing algorithms have simple decision-making mechanisms and low flexibility, making them unable to adapt to the rapid and dynamic changes in UAV network topology; while intelligent routing algorithms typically require substantial computing and communication resources, making them difficult to apply effectively in resource-constrained UAV networks. To address this contradiction, this invention designs an enhanced twin technology and constructs a novel intelligent routing mechanism for UAV networks based on enhanced twins. It focuses on resolving the deviation in routing decisions caused by synchronization errors between the physical space and the enhanced twin space of the UAV network, and dynamically and intelligently optimizes the routing pool according to the network environment, achieving intelligent adaptive capability of the routing mechanism to the external environment. The technical solution is described in detail below:

[0021] 1. Scene Description

[0022] like Figure 1 As shown, assuming all drones in physical space fly at the same altitude, consider a range Z = [x min x max ]×[y min y max Within a two-dimensional region, there is a group of moving unmanned aerial vehicles (UAVs), denoted as U = {u1, ..., u2}. n}, where x max and y max These are the maximum boundary values ​​of the region in the x and y dimensions, respectively. Each drone u i It corresponds to a unique ID value (i).

[0023] Assuming that each node in the network has the same effective communication range, and that the effective communication distance of the UAV is represented as d max . When u i and u j distance d ij ≤d max When, it means u i and u j If two nodes can communicate with each other and are each other's neighbors, the link between them is labeled as e. ij =1. Based on the connectivity between drones, u i The set of adjacent nodes can be represented as N i ={u j |e ij =1}, where j∈{1,2,...,n} and i≠j. A set of several communication links in the network is E={eij Let N = 1 | i, j ∈ {1, 2, ..., n} and i ≠ j}, and let G(U, E) represent the UAV network topology. To adapt to dynamic changes in the network topology, each node periodically sends detection information to discover neighboring nodes within its communication range. The node information N in the network... info and business flow information F info The data is periodically uploaded to the twin space to dynamically update the state of the virtual drone network within the twin space. Therefore, twin nodes can continuously update their neighbor tables, thereby maintaining the overall network topology.

[0024] In a twin space, the routing process of an unmanned aerial vehicle (UAV) ad hoc network over a period of time is simulated using collected network information. To adapt to dynamic changes in the network environment and fully leverage the advantages of different routing algorithms, m routing algorithms are simultaneously simulated, denoted as RA. P ={ra1, ra2, ..., ra m These algorithms can be based on different principles, such as topology, geographic location, and quality awareness, or they can consider decision-making algorithms based on different network attributes. Here, m is an integer greater than or equal to 2, typically taking a value between 3 and 6. By comparing the comprehensive evaluation indicators obtained from simulating multiple routing algorithms in twin space, the best-performing routing algorithm is selected, and the routing decision results of the pre-simulation process are fed back to the UAV entity.

[0025] The collected network information specifically includes: node location information, link connection status, node queue information, and service load information.

[0026] 2. Error-aware intelligent routing mechanism based on augmented twins

[0027] Because augmented twins require time to process drone synchronization data, the highly dynamic drone network may experience topology changes, communication link quality changes, and network connectivity changes during this period. Consequently, the routes selected by the augmented twin may deviate from those in the real world. This invention addresses the synchronization error problem between the augmented twin and the drone, and designs an error-aware intelligent routing technology based on augmented twins, such as... Figure 2 As shown, firstly, the routing algorithm is initially evaluated in the current network environment based on different routing metrics using the results of enhanced twin simulation routing. Then, considering the synchronization error between the enhanced twin and the UAV, an error correction model is designed to ensure that the optimal routing method selected by the enhanced twin maintains high adaptability to the dynamic environment of the UAV.

[0028] (1) Routing evaluation metrics

[0029] Assuming that during the data packet transmission process of the UAV, the p-th data packet originates from the source node u sTransmitted to destination node u d A multi-hop path is represented as P s，d The set of nodes on the path is represented as U. p ={u s =u p0 u p1 , ..., u pz =u d}, where u pi (i∈{0,1,...,Z}) represents the transmission path P s，d The i-th hop drone node. To evaluate the performance of routing algorithms, multiple performance metrics are typically considered. Common performance metrics for evaluating routing algorithms include the following aspects:

[0030] (a) Average end-to-end delay

[0031] End-to-end delay refers to the time required for a data packet to travel from the source node to the destination node, including queuing delay, processing delay, transmission delay, and propagation delay of all nodes along the transmission path. The end-to-end delay of the p-th data packet is expressed as:

[0032]

[0033] in, They are nodes u pi Queuing delay, processing delay, and transmission delay. For node u p(i-1) to node u pi The propagation delay.

[0034] The average end-to-end delay is the average of the sum of the end-to-end delays of all data packets received by the destination node, expressed as:

[0035]

[0036] Where, N p D represents the number of data packets successfully received. p This represents the end-to-end delay of the p-th data packet.

[0037] (b) Package delivery success rate

[0038] In dynamic drone networks, data packets may be lost during forwarding due to factors such as packet timeouts, exceeding the maximum hop count, or node queue overflow. Packet Success Rate (PDR) is defined as the ratio of the number of packets received by the destination node to the number of packets sent by the source node, expressed as follows:

[0039]

[0040] Where, N a N represents the total number of data packets sent. l This represents the number of lost data packets. Packet delivery success rate reflects the connectivity and stability of a route and is an important indicator of the performance of a routing algorithm.

[0041] (c) Throughput

[0042] System throughput is defined as the number of bytes of data packets transmitted per unit of time. Time period T end -T start The throughput within is expressed as:

[0043]

[0044] Where, size p Let p be the size of the p-th data packet.

[0045] (d) Average number of jumps

[0046] Average hop count is a commonly used routing performance metric, referring to the average number of hops that all data packets take during transmission. Generally, a lower average hop count indicates a shorter route and faster data transmission speed.

[0047]

[0048] Among them, H p Let p be the number of hops in the transmission of the p-th data packet.

[0049] (e) Data normalization

[0050] When evaluating drone routing performance, among individual metrics, lower latency indicates a better routing algorithm, while higher packet delivery rate and throughput represent superior algorithm performance. Since these four metrics have different dimensions, they are normalized using the sigmoid function, as shown below:

[0051]

[0052] Where c1>0, c2<0, c3<0 are used to adjust the slope of the sigmoid function, T max and PDR max These represent the maximum allowable latency and maximum allowable packet loss rate for data service transmission, respectively. max This represents the maximum number of hops during transmission.

[0053] (2) Factors affecting index error

[0054] Consider the difference δ between the physical entity and the twin space, which may stem from the data interaction delay t between the twin and the physical world.p-at Data loss may occur when data is uploaded from the physical world to the twin world. p-at And the inaccuracy of data in the twin simulation process. p-at Multiple factors, including the differences between the physical world and the twin world, have varying impacts on the various routing algorithms simulated in the twin space. The following analysis will examine t... p-at , l p-at and ia p-at Impact on several typical UAV network routing algorithms.

[0055] (a) Interaction latency

[0056] Data generated in the physical world is transmitted to the twin world at intervals on the order of seconds. The interaction delay between the physical space and the twin space is represented by the moment t when the data is generated in the physical world. p The moment t' of updating the state with the twin space at The difference between them is denoted as t. p-at =t' at -t p In this process, the drone periodically transmits data to the twin space, and the twin receives and analyzes this data to update the model.

[0057] When routing changes occur in the physical space, information in the twin space may not immediately reflect these changes due to interaction latency. This interaction latency affects different routing algorithms differently, primarily depending on the routing establishment process, route update method, and maintenance method after route failure. Route establishment in UAV networks involves establishing proximity connections via wireless communication, discovering the network topology and neighboring nodes, exchanging routing information to build and update the routing table, considering mobility management and link quality assessment, and ensuring the stability and optimization of communication paths between nodes. Route updates continuously detect topology changes and update the routing table in real time based on mobility, signal strength, etc., ensuring the network adapts to dynamic environments. Maintenance after route failure involves identifying faulty nodes, quickly repairing paths or recalculating routes, ensuring the continuity of information transmission, and guaranteeing network security and stability.

[0058] Routing establishment, updates, and maintenance all rely on real-time information within the network, such as node location, node speed, connection quality, and status information. The information update cycle determines when this critical information is refreshed and updated, directly impacting the accuracy and timeliness of routing. An appropriate information update cycle ensures that routing decisions are based on the latest and most accurate data, thereby improving the performance and reliability of the UAV network. Therefore, considering the use of an information update cycle is crucial. This is used to measure the impact of interaction latency between physical space and twin space on different routing algorithms. The longer the information update cycle required by the routing algorithm, the stronger its adaptability, and the smaller the impact of interaction latency on the algorithm's operation.

[0059] The initial active routing pool is determined using historical experience, and based on that experience, 3-6 routing algorithms that are best suited to the current network environment are selected.

[0060] In the route establishment process, topology-based routing can be divided into two basic types: proactive and reactive. Proactive routing algorithms, such as OLSR and DSDV, are characterized by periodically maintaining routing information, keeping the routing table up-to-date even when there is no communication demand. Therefore, interaction latency has a relatively small impact on proactive routing. In contrast, reactive routing algorithms, such as AODV and DSR, only trigger the route discovery process when communication needs to be established, effectively reducing the use of network resources, but introducing greater latency during communication establishment. Therefore, reactive routing is more sensitive to interaction latency. Geographically based routing, such as GPSR, uses the geographical coordinates of nodes to determine the path of data packet transmission, thereby realizing data transmission and having advantages in adapting to dynamic environments. Therefore, geographically based routing is less sensitive to interaction latency than topology-based routing. In addition, flooding does not rely on prior routing information and is suitable for dynamic network environments and situations where the network topology cannot be predicted in advance. In flooding, after a node receives a data packet, it forwards it to all neighboring nodes, thereby covering the entire network.

[0061] Therefore, the information update cycle of the routing algorithm The method of route establishment is a major factor in measuring the impact of interaction latency on routing algorithms. Interaction latency affects the i-th routing algorithm ra in parallel simulations. i Influence values ​​of ∈{OLSR, AODV, GPSR, FLOOD, ...} Defined as:

[0062]

[0063] Where α1+β1=1, the two are used to adjust the influence weight of Hello message period and route discovery method. In this invention, α1=β1=0.5 is taken. The impact of interaction latency on routing algorithms under different route establishment methods. The larger the value and The smaller the value, The smaller the value, the less impact the interaction latency has on the routing algorithm.

[0064] The following analysis uses four typical routing algorithms—OLSR, AODV, GPSR, and flooding—as examples to illustrate the impact of interaction latency on different routing algorithms. OLSR's information updates are closely related to the sending of Hello messages; therefore, the information update period can be set to the Hello message sending period, typically 2 seconds. AODV's information updates are mainly triggered when a change in link state is detected; therefore, the information update period can be set to the link state detection time interval, typically 3 seconds. In UAV networks, the GPSR geographic information beacon update period is set to 1.5 seconds. Flooding, because it does not require information updates, has its information update period set to infinity.

[0065] Topology-based routing relies on global network topology information and is sensitive to topology changes, potentially leading to performance degradation in highly dynamic environments. In contrast, geographic-based routing utilizes the geographical location of nodes, providing more dynamic adaptability. Therefore, interaction latency has a smaller impact on geographic-based routing than on topology-based routing. In topology-based routing, proactive routing periodically maintains routing information and can directly respond to current communication needs, while reactive routing only begins pathfinding when data transmission is required. Therefore, interaction latency has a smaller impact on OLSR than on AODV. Flooding does not require a route discovery process and is therefore almost unaffected by interaction latency. Based on the analysis of the impact of interaction latency on different routing methods, the impact values ​​of route discovery methods OLSR, AODV, GPSR, and FLOOD are set as follows:

[0066] According to equation (10), the impact of interaction delay on the four types of routing is obtained as shown in Table 1:

[0067] Table 1 Impact Values ​​of Interaction Delay

[0068]

[0069] The impact of interaction latency on routing It is divided into multiple levels according to the increment Δ. For interaction latency to ra i The impact levels are shown in Table 2:

[0070] Table 2 Impact Level of Interaction Latency

[0071]

[0072] Here, Δ = 0.50, so the impact level of interaction latency on OLSR, AODV, GPSR, and FLOOD can be set as follows:

[0073]

[0074] (b) Interactive packet loss

[0075] During the process of transmitting data from the physical world to the twin world, data loss may occur due to various reasons such as network congestion. This loss affects the integrity of the twin data and may lead to discrepancies in the update states of different twins, thus impacting the accurate simulation and reflection of the physical world.

[0076] When data loss occurs during transmission from the physical world to the twin world, it is likely due to network resource constraints. Different routing algorithms may be at odds during route selection and data transmission. i The average number of forwardings of routing control packets and data packets is defined as the corresponding network overhead. By calculating the average number of forwards, the load on nodes in the network and the impact of topology on data transmission can be assessed, thus measuring the impact of packet loss caused by the interaction between twin space and physical space on different routing algorithms. OLSR is a link-state algorithm that maintains the topology information of the entire network and periodically exchanges control messages to update the routing table. While MPR optimizes information propagation and reduces the transmission of some redundant data, frequent control message exchanges lead to significant network overhead and bandwidth consumption as the network size increases. In contrast, AODV is an on-demand distance vector algorithm that initiates route discovery only when needed, thereby reducing communication overhead, especially suitable for situations where inter-node communication is infrequent. However, in environments where the network topology changes frequently, AODV may increase overhead due to frequent route requests. GPSR is a stateless algorithm whose routing overhead mainly involves maintaining and updating node location information to select the nearest neighbor node as the next hop based on a greedy algorithm. Its routing overhead is relatively low because it does not require a global routing table, but only relies on local neighbor information and node locations. However, in highly dynamic or dense networks, frequent updates to node locations may increase control message and computational costs. Flooding, as a simple and direct broadcasting method, sends data packets to all neighboring nodes. However, this method incurs significant routing overhead, especially in large-scale networks, resulting in substantial redundant data transmission. Choosing a suitable algorithm should consider network characteristics and resource consumption.

[0077] When physical space and twin space interact, leading to data loss, end-to-end routing and hop-by-hop routing exhibit different characteristics. End-to-end routing, such as OLSR and AODV, typically maintains global routing information across the entire network. When data loss occurs during end-to-end transmission, these algorithms may trigger global route updates, resulting in flooding of control messages and recalculation of paths across the entire network, thus increasing network overhead and latency. In contrast, hop-by-hop routing strategies offer greater flexibility and adaptability. In the event of data loss, hop-by-hop routing, such as flooding and GPSR, allows for local route adjustments. Flooding, as a simple hop-by-hop routing algorithm, retransmits data packets but may cause excessive redundancy and flooding storms in the network, increasing bandwidth pressure. GPSR, on the other hand, selects hop-by-hop routes based on node locations and may attempt to choose new paths after data loss, thus being influenced by node location information. The impact of interactive packet loss on routing algorithms under different routing selection methods is denoted as […].

[0078] Therefore, the overhead of the routing algorithm and the route selection method are the main factors in measuring the impact of packet loss on the routing algorithm. i Influence values ​​of ∈{OLSR, AODV, GPSR, FLOOD, ...} Defined as:

[0079]

[0080] In the formula, α² + β² = 1, and these two values ​​are used to adjust the influence weights of network overhead and routing selection method. In this invention, α² = β² = 0.5. When The value and The smaller the value, The smaller the value, the less impact packet loss on the routing algorithm. (This is related to computation.) Similarly, it can determine the impact of packet loss on routing. Get the interaction packet loss pair ra i Impact level

[0081] The following analysis uses four typical routing algorithms—OLSR, AODV, GPSR, and flooding—as examples to examine the impact of packet loss on different routing algorithms. During transmission, GPSR, AODV, OLSR, and flooding exhibit significant differences in the average number of forwards for all data packets. GPSR uses a greedy algorithm to select the nearest neighbor for packet transmission, typically resulting in a low average number of forwards, making it suitable for small-scale, high-density self-organizing networks. AODV employs on-demand routing, establishing routing tables only when needed, resulting in a relatively low average number of forwards, making it suitable for medium-sized networks. OLSR, based on link-state routing, optimizes routing table maintenance, achieving a certain level of efficiency, and is suitable for relatively static networks. Flooding is a simple and intuitive broadcasting method, but its average number of forwards is high, potentially leading to network storms, especially in large networks. Therefore, the choice of a suitable algorithm depends on the network size and requirements, seeking a balance between performance and network utilization efficiency.

[0082] End-to-end routing may be more affected by global route updates and control message exchanges, while hop-by-hop routing is more local and flexible, but may also introduce challenges such as flooding effects or high dependence on node locations. Based on the analysis of the impact of packet loss on different routing methods, the impact values ​​of routing methods OLSR, AODV, GPSR, and FLOOD are set as follows:

[0083] (c) Inaccurate data

[0084] When the data required for routing decisions is inaccurate in the twin space, the root causes include measurement errors, transmission distortion, and model limitations. First, data from physical world sensors may contain errors due to limitations in their measurement accuracy. Second, bit errors may occur during data transmission from the physical world to the twin world, leading to data inaccuracies. Finally, the complexity, randomness, and uncertainty of channels in the real environment may not be fully and accurately simulated or reflected in the twin space, resulting in discrepancies between simulated and real data. These inaccuracies between the physical and twin worlds can cause differences in routing decision performance between the twin space and the physical network. For network topology-based algorithms such as OLSR and AODV, inaccurate data can hinder accurate network topology establishment and maintenance, affecting routing selection and update processes, thus reducing communication efficiency. Furthermore, if the data in the twin space is inaccurate, AODV may make errors in routing decisions, leading to unnecessary route discovery processes, increasing control message transmission overhead, and impacting algorithm performance. For routing algorithms using greedy algorithms, such as GPSR, inaccurate data can cause nodes to make incorrect decisions when choosing the next hop, affecting algorithm performance and potentially leading to the selection of non-shortest paths, thus impacting overall routing performance. Inaccurate twin data can also lead to incorrect guidance of the physical network, resulting in the transmission of more redundant routing control packets. This not only increases the network load but can also reduce overall network efficiency.

[0085] Different routing algorithms differ in their adaptability, information acquisition methods, and routing table maintenance when making route selections. Therefore, they use different information for decision-making. (The last sentence, "Rabbit algorithm ra," appears to be a separate, unrelated statement and is left untranslated.) i The impact of inaccurate data on the routing algorithm under the decision-making method is denoted as . Furthermore, in the twin space, the difference Δp = pp′ between the error-adjusted comprehensive evaluation value p′ in the twin space and the comprehensive evaluation value p in the physical space reflects the degree of performance impact caused by inaccurate data in the twin space and the physical space. Therefore, according to The difference Δp between the twin-space comprehensive evaluation and the data inaccuracy is used to measure the impact of data inaccuracy on the routing algorithm ra. i Influence values ​​of ∈{OLSR, AODV, GPSR, FLOOD, ...}

[0086]

[0087] The following analysis uses four typical routing algorithms—OLSR, AODV, GPSR, and flooding—as examples to illustrate the impact of inaccurate data on different routing algorithms. OLSR employs a link-state routing algorithm, maintaining a global topology map by periodically exchanging link-state information. In routing decisions, the main information used includes the connection status between nodes, link quality, and topology. Inaccurate uploaded location information can lead to misjudgments of link-state information, affecting routing table updates and routing decisions. AODV is an on-demand distance vector algorithm. In routing decisions, the main information used is the distance between nodes and the number of hops to the target node. Inaccurate uploaded location information can lead to misjudgments of distance between nodes, affecting the accuracy of route selection. GPSR directly uses node location information for routing decisions. At each step, it locally selects the next hop based on the node's current location and neighbor information to find the nearest neighbor node, without considering the global network state. Inaccurate uploaded location information can have a significant impact on GPSR because it directly affects the accuracy of routing decisions. Inaccurate node location information can lead to incorrect next-hop selection, or even route failure or loops. Flooding is a simple broadcast routing method that achieves message delivery by broadcasting data packets to all neighboring nodes. Therefore, inaccurate uploaded location information has a relatively small impact on the flooding algorithm. Even with inaccurate location information, the flooding algorithm can still deliver data packets to all neighboring nodes, but it may lead to an expanded broadcast range or unnecessary duplicate transmissions, thus affecting network efficiency. Based on the analysis of the impact of inaccurate data on different routing decisions, the impact values ​​of routing selection methods OLSR, AODV, GPSR, and FLOOD are set as follows:

[0088] In UAV ad hoc networks, mobile nodes perform tasks in complex and dynamic environments. Selecting a suitable data routing scheme is crucial for achieving efficient and timely data transmission. This invention's routing evaluation model first normalizes the simulation results of various performance indicators. Next, it adjusts the normalized results of each indicator using influencing factors. Finally, it linearly weights the adjusted indicator results according to the indicator weights for different service requirements to obtain a comprehensive evaluation value for each routing algorithm. Multiple routing algorithms are simulated in parallel in a twin space and compared using the evaluation model. The routing algorithm with the highest score is fed back to the physical space, providing a basis for optimizing UAV network data routing to reduce communication latency and improve packet arrival rate and network throughput. The comprehensive evaluation value obtained from the routing evaluation model will provide reliable guidance for selecting the optimal routing strategy in the twin space, enabling efficient routing selection in complex environments for UAV ad hoc networks.

[0089] (3) Error correction model

[0090] In the parallel simulation process in the twin space, there are m evaluation objects, which are m routing algorithms simulated in parallel in the twin space, denoted as RA. P ={ra1, ra2, ..., ra m When evaluating the performance of m routing algorithms, a challenge arises: differences exist between the physical space and the twin space, meaning the performance results obtained from twin space simulations may not be entirely accurate, and these differences affect different routing algorithms to varying degrees. Therefore, to ensure the reliability of the evaluation results, it is necessary to comprehensively consider the impact of these differences on different routing algorithms and adjust the twin space simulation results to obtain a more objective and fair performance evaluation.

[0091] Through parallel simulation, performance data of various routing algorithms in different aspects can be obtained, forming a comprehensive result matrix, which provides a foundation for further routing performance evaluation and comparison. For the i-th routing algorithm ra... i Construct a vector This includes two parts. The first part is the set of normalized simulated values ​​of a constant route evaluation indicators, denoted as... The second part discusses the impact of three factors—interaction delay between the twin space and physical space, data packet loss, and data inaccuracy—on the routing algorithm Ra. i The set of influence factors, denoted as The evaluation matrix SR = {SR1, SR2, ..., SR3} is composed of m evaluation objects and a+3 evaluation indicators. m}=(sr ij ) m×(a+3) , represented as:

[0092]

[0093] Due to interaction delays, data loss, and data errors between the twin space and the physical space, the metrics obtained from routing simulations in the twin space may be inaccurate, thus affecting the accurate evaluation of routing performance and the effective decision-making for routing algorithm selection. The differences between the twin space and the physical space have varying degrees of impact on different routing algorithms. To more objectively and fairly evaluate the performance of different routing algorithms, a set of influencing factors is used. To adjust the performance results of different routing algorithms, thereby reducing the impact of differences on the performance evaluation of routing algorithms. Comprehensive utilization The elements in the formula are used to construct a regulating factor δ. p-at This indicates the degree of adjustment made to the simulation results of different routing algorithms.

[0094] Taking interaction latency as an example, this paper analyzes the relationship between the differences between physical space and twin space and the adjustment factor, and examines the impact of interaction latency on the routing algorithm ra.i The degree of influence is greater than that of ra k The degree of influence is higher, that is, sr i(a+1) >sr k(a+1) At that time, ra i The adjustments to various performance indicators are significant, at which point ra i The larger the value of the adjustment factor, the better.

[0095] Furthermore, the correlation between the adjustment factor and the influence of interaction packet loss and data inaccuracy is similar to the analysis of interaction latency described above, but there are some differences in their impact on the trend of adjustment factor changes. For interaction latency and interaction packet loss, when their influence levels are low, the routing performance of the twin space and the physical space is relatively consistent, so the adjustment factor initially increases slowly with the increase of influence level. As the influence level continues to increase, the performance differences between the twin space and the physical space gradually become more obvious, so the adjustment factor also increases rapidly. However, when the influence level reaches a certain level, the adjustment factor tends to saturate, and the rate of increase slows down.

[0096] In the initial stages of the escalating impact of data inaccuracy, the inaccuracy significantly increases the performance difference between the twin-space routing simulation and the physical space routing process. Therefore, the adjustment factor exhibits a rapid increasing trend to adjust the simulation results to accommodate the differences. At this point, the rapid increase in the adjustment factor may reflect sensitivity to inaccuracy, with the evaluation module attempting to adapt quickly to arrive at an objective evaluation result. As the impact level further increases, the adjustment factor gradually approaches saturation because the room for adjustment becomes limited when facing extremely high uncertainty. At this stage, the growth of the adjustment factor slows down, indicating that the evaluation module's response to further inaccuracy is gradually weakening, or that adjustment has reached a certain limit. This saturation phenomenon reflects the limitation of the evaluation's ability to adjust in extreme situations. Based on the above analysis, the routing algorithm ra... i The regulating factor is modeled as follows:

[0097]

[0098] Here, 0≤k1,k2≤1 and γ,ε,μ≥0 are variable parameters used to adjust the linear and nonlinear effects of interaction latency, packet loss, and data inaccuracy on the adjustment factor. Packet loss during the interaction process has a combined impact on interaction latency and data inaccuracy; therefore, the adjustment factor is designed to multiply the impact of packet loss by the linearly weighted result of the impact of interaction latency and data inaccuracy. When the errors caused by interaction latency, packet loss, and data inaccuracy are small, i.e., the data deviation between the twin space and the physical space is small, the performance evaluation of the routing algorithm is relatively less affected by the errors. However, as the errors increase, the data differences between the twin space and the physical space gradually increase. This is because, with the increase in interaction latency, packet loss, and data inaccuracy, the network state and topology in the physical space may have changed, and these changes have not yet been reflected in the twin space. Therefore, the data synchronization between the twin space and the physical space gradually decreases, and the routing algorithm may experience performance degradation or decision failure under such circumstances. As the impact of errors on the performance evaluation of the routing algorithm increases, the adjustment factor also increases accordingly.

[0099] Based on simulated data, adjustment factors are used to compensate for the impact of differences on various routing algorithms, thus obtaining a more objective and impartial routing evaluation model. The routing algorithm ra is then used. i Regulatory factors Adjust ra i The simulation results yielded the final performance index sr′ ij The expression is as follows:

[0100]

[0101] Where 0≤k3 and k4≤1 are variable parameters used to adjust the normalized performance index results and the weight ratio of the adjustment factor.

[0102] The adjusted results are linearly weighted to obtain ra. i Comprehensive evaluation value The expression is as follows:

[0103]

[0104] Among them, w j Let w be the weight coefficient of the j-th indicator. The set of weight coefficients for all performance indicators is represented as w = {w_j}. j}, where j∈{1,2,…,a}. In UAV networks, the relative importance of commonly used routing evaluation metrics varies depending on business requirements. In practical application scenarios, considering the different importance of various performance metrics, the weight coefficients of each metric can be adjusted according to business needs to better meet specific application requirements. For example, in emergency rescue, it is necessary to quickly acquire and transmit real-time video or sensor data, making latency a relatively important factor. In this case, the weight coefficient of latency can be set higher than that of other performance metrics. When performing tasks that require ensuring data integrity or transmitting critical data such as control commands, packet loss rate becomes a key metric. In addition, in applications such as large-scale data transmission or image acquisition and processing, throughput directly affects the efficiency of task completion. Adjusting the weight coefficients of corresponding metrics according to business needs makes UAV networks more flexible in adapting to various application scenarios.

[0105] 3. Connectivity-based Federated DQN active routing pool intelligent optimization technology

[0106] In addition to an active routing pool containing a single routing algorithm, a routing algorithm library with a wider range of algorithms exists within the virtual space. Routing algorithms better suited to the current environment are selected from this library to replace those in the active routing pool that are no longer suitable. This ensures that the routing algorithms in the active pool are updated online, adapting to the highly dynamic and changing conditions of the UAV network. It is precisely this dynamic update technology for the active routing pool that eliminates the need for parallel simulation of all routing algorithms in the library. While maintaining the high efficiency of the augmented twin-based intelligent routing selection technology, this significantly reduces the number of parallel simulation algorithms and saves computational overhead.

[0107] like Figure 3As shown, the routing algorithm library mainly includes four categories of algorithms: topology-based, geolocation-based, bio-inspired, and reinforcement learning-based. Topology-based algorithms are suitable for scenarios with relatively stable and slowly changing network structures, particularly for long-duration tasks or fixed-area monitoring. Geolocation-based routing algorithms are suitable for environments that require consideration of node location information and can fully utilize the geographic location information of UAVs, especially suitable for tasks requiring coverage of large areas. For highly dynamic networks, traditional routing algorithms cannot adapt to real-time and complex network changes in a timely manner. The introduction of intelligent improvement algorithms provides more choices and more flexible solutions for wireless ad hoc networks. Bio-inspired algorithms draw on the characteristics of evolution, group behavior, and information transmission in the biological world, enabling them to adapt to dynamic environments and provide stable and efficient routing schemes based on node movement and changes in network topology. Reinforcement learning-based algorithms can dynamically adjust routing strategies based on environmental feedback, continuously optimizing routing strategies through interactive learning with the environment, making them suitable for application scenarios that require rapid adaptation to environmental changes and real-time optimization of communication performance.

[0108] In the augmented twin's virtual space, each drone's augmented twin node deploys a local DQN model, which is trained based on its own perception of environmental information. Additionally, a global DQN model is deployed, obtained by aggregating all local DQN models across the network, and updates the active routing pool based on network-wide environmental information. First, a Markov decision process is constructed, represented as a quadruple.<S,A,R,γ> The elements in the diagram represent the state space, action space, reward function value, and discount factor of the environment information, respectively. The DQN algorithm updates the state-action value function based on the experience of each agent to guide them in making better decisions. Through continuous interaction and learning with the environment, the agent gradually learns strategies for selecting the optimal routing algorithm in different situations, thus achieving adaptive optimization of the network routing algorithm. The proposed active routing pool update framework based on DQN consists of an evaluation Q-network, a target Q-network, and an experience replay pool. The evaluation Q-network is used to update the state-action value function based on the state s. t Generate Q-values ​​for each action; the target Q-network is used to mitigate the overestimation problem during the evaluation Q-network training process; the experience pool is used to store experience (s t a t r t s t+1 ).

[0109] (1) State space

[0110] Because each drone's virtual twin is deployed with its own DQN and trained independently, the drone's... iThe state space of the local DQN model deployed on it. This refers to the local state space obtained by the UAV after processing the local environmental information it can observe. The observation range of each node is set to include its own state and the state information of all its one-hop neighbors. i In terms of its state space Specifically, it includes u i And all one-hop neighbors N i All nodes {u i N i The calculated local density Topological dynamics Link stability and network congestion

[0111]

[0112] Meanwhile, for the global DQN model, its state space s o This refers to the environmental status information comprised of all drones on the network, specifically including the overall density (De). o Topology Dynamics (TD) o Link stability LS o and network congestion level NL o .

[0113] The density of nodes (De) reflects the connectivity and distribution of nodes in the network; the topology dynamics (TD) represents the movement of nodes and the disconnection of links; the link stability (LS) uses received power to describe the stability and quality of network links; and the network congestion (NL) represents the load in the network through the node queue situation.

[0114] (2) Action space

[0115] Action space A is derived from the routing algorithm library RA. m ={or1, or2, ..., or M Choose a routing algorithm from the given list. Therefore, the action space can be represented as:

[0116] A = {or1, or2, ..., or...} M} (18)

[0117] Furthermore, to prevent DQN from getting trapped in local optima during training, this invention employs a greedy strategy to optimize action values ​​during training. During training, each UAV's local DQN model will explore random actions with a probability of ∈1, and select the action with the highest Q-value with a probability of (1-∈1). The specific action function is as follows:

[0118]

[0119] Where rand is a random number (rand∈[0,1]), Random(A) represents random exploration, taking a random action in the action space A, Q(s i a i ;w i ) indicates that when the parameter is w i Evaluation of the state s in the Q network i The following measures were taken: i The reward value of the action.

[0120] Similarly, when using the DQN global model to select the optimal routing algorithm, exploration is performed with a probability of ∈2, while actions that maximize the reward are performed with a probability of (1-∈2). The specific action function is expressed as:

[0121]

[0122] Where Q(s) o a o ;w o ) indicates that when the parameter is w o In the global evaluation Q-network, the global environment state is s o The following measures were taken: o The global reward value of the action.

[0123] (3) Reward and punishment functions

[0124] For drones i If in environment S i ={De i TD i LS i NL i The following action is to select routing algorithm a. i Meanwhile, if the DQN global model selects routing algorithm a based on the global network environment status information... o Then the routing algorithm a will be used in the virtual space. o Perform a simulation and obtain the global reward value according to equation (16).

[0125]

[0126] in This represents the simulated score of the k-th algorithm in the active routing pool. Representing algorithm a o The score after simulation This represents the simulated score of the routing algorithm that was replaced in the active routing pool.

[0127] So, drones i Decision-making action a i The corresponding reward function can be the global action 'a' made based on the DQN global model. o and the corresponding global reward value R o To determine. When local action a i With global action a o When they are the same, it can be determined based on algorithm a. o The simulated return value R o Give a larger reward value; while when the local action a i With global action a o If they are different, then a penalty value for a missed action is given, i.e.:

[0128]

[0129] Where α is the weight hyperparameter, R p It is the penalty value for missing an action.

[0130] like Figure 4As shown, the system updates the active route pool in two ways: triggered updates and periodic updates. Triggered updates occur during parallel simulations when the algorithm with the highest overall score in the active route pool falls below a system-defined threshold. This triggers an active route rotation mechanism with a minimum trigger interval to prevent cyclical updates. In addition, the active route pool is periodically updated every fixed period T to maintain its adaptability to environmental changes. After enabling updates to the active routing pool, the current global environment-aware information is first input into the DQN global model for decision-making, outputting the current optimal action, which is the best routing algorithm in the routing algorithm library other than those in the active routing pool. Then, the optimal routing algorithm and all algorithms in the original active routing pool are simulated and evaluated in parallel to obtain a comprehensive evaluation score. If the score of the optimal routing algorithm is higher than that of the lowest-scoring algorithm in the original active routing pool, the algorithm with the decision is used to replace the lowest-scoring algorithm in the original active routing pool; otherwise, no update is performed. The typical value of threshold is between 0.3 and 0.6, and the typical value of T is between 50 and 100. Both values ​​are determined based on the adaptability of the routing algorithms in the current active routing pool to the network. In the augmented twin's virtual space, drone nodes acquire the current network state through communication and perception with surrounding nodes. After processing the network environment state information, the agent selects a routing algorithm from the routing algorithm library as an action based on the DQN network. The selected routing algorithm is then simulated and comprehensively scored within the local network to obtain the reward value for the action and the next state to proceed to. This process is used to train and update the DQN model parameters. The pseudocode for this algorithm is as follows:

[0131]

[0132] Number of iterations The typical value is 10000; This represents the number of time slots in one training round, typically 7200. In practical applications, its value is determined based on task requirements.

[0133] In federated learning, the parameter server that performs model aggregation is a virtual central node in the virtual space. Similar to the virtual twins of each drone, the actual computation and communication of this parameter server are carried out on a server in the physical space, and it interacts with the virtual drone twins through this server. Therefore, the communication capability between nodes is greatly enhanced during federated learning.

[0134] In typical federated learning, model parameters uploaded from each node have the same weights. However, each node, due to its different network environment, obviously has varying degrees of influence on the global routing algorithm selection. A node's connectivity represents its association with other nodes, thus it can also represent its impact on overall network performance. When the parameter server updates the model, connectivity-based weights are used to adjust the global federated model aggregation function. Specifically, for drone node u... i For example, if its connectivity is deg i Then the aggregation function of the global federated model based on connectivity is:

[0135]

[0136] Furthermore, when updating the active route pool, it is necessary to select the best routing algorithm from the routing algorithm library that is not in the active route pool for replacement. Therefore, this invention uses a mask vector to mask algorithms already in the active route pool. The mask vector is a vector with a length equal to the action space size M, where each element represents an algorithm (i.e., a routing algorithm) in the routing algorithm library. Each element has a value of only 0 or 1; correspondingly, a value of 0 means that the routing algorithm is masked, and a value of 1 means that the routing algorithm can be selected. Therefore, when updating the active route pool, it is necessary to select a routing algorithm from outside the active route pool to replace the worst-performing routing algorithm in the current active route pool. Thus, firstly, the bits corresponding to the m algorithms already in the active route pool in the mask vector are set to 0, and all other bits are set to 1, that is, the selection scope is limited to algorithms not in the active route pool; algorithms already in the active route pool do not need to participate in this selection.

[0137] M is an integer greater than m, usually a value between 6 and 20. The specific value of M can be determined according to the application scenario and requirements.

[0138] Therefore, the action of updating and replacing the active route pool according to the global model can be represented as follows:

[0139]

[0140] in This is the mask vector. By multiplying the mask vector by the Q-value vectors of each action, the routing algorithm with the highest benefit value can be selected from the routing algorithm library that masks the existing algorithms in the active routing pool, and the worst performing algorithm in the active routing pool can be replaced.

[0141] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Various modifications and variations can be made to the invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the scope of the claims of the invention.

Claims

1. A self-evolving routing method for UAV networks based on enhanced twins, characterized in that, The enhanced twin UAV network includes: a drone, a server, and a virtual space running on the server. The virtual space includes several enhanced twins, each corresponding to a drone. The method specifically includes the following steps: S1. The drone transmits network environment data to its corresponding enhanced twin; S2. Enhanced twin performs routing simulation based on received network environment data to obtain the corresponding indicator data for different routing algorithms in the active routing pool. S3. Adjust the data of each indicator according to the error evaluation model to obtain the scores of different routing algorithms in the active routing pool; the error evaluation model is expressed as: Among them, sr′ ij Represents the i-th routing algorithm ra i The adjusted j-th indicator, sr ij Represents the i-th routing algorithm ra i The j-th indicator before adjustment, where k3 and k4 represent variable parameters, 0 ≤ K3, k4 ≤ 1. Represents the i-th routing algorithm ra i The corresponding adjustment factor; The i-th routing algorithm ra i The expression for the corresponding adjustment factor is: Where k1, k2, γ, ε, and μ represent variable parameters, 0 ≤ k1, k2 ≤ 1, γ, ε, μ ≥ 0; t p-at Indicates the interaction latency, sr i(a+1) Indicates the interaction delay impact factor, l p-at Indicates the packet loss rate during interaction, sr i(a+2) The factor representing the impact of packet loss rate in interactive communication, ia p-at This indicates that the data is inaccurate, sr i(a+3) Indicates factors that affect the accuracy of data; S4. Select the optimal route for each drone based on the scores of different routing algorithms in the active routing pool.

2. The self-evolving routing method for UAV networks based on enhanced twins according to claim 1, characterized in that, The metrics data include commonly used performance metrics for evaluating routing algorithms and interaction factors between drones and augmented twins.

3. The self-evolving routing method for UAV networks based on enhanced twins according to claim 2, characterized in that, Common performance metrics for evaluating routing algorithms include: average end-to-end latency, packet delivery success rate, throughput, and average hop count.

4. The self-evolving routing method for UAV networks based on enhanced twins according to claim 3, characterized in that, The interaction factors between drones and augmented twins include: interaction latency, interaction packet loss rate, and data inaccuracy.

5. The self-evolving routing method for UAV networks based on enhanced twins according to claim 4, characterized in that, In step S3, by setting weight coefficients for each adjusted indicator data and performing linear weighting, the scores of different routing algorithms in the active routing pool are obtained.

6. The self-evolving routing method for UAV networks based on enhanced twins according to claim 5, characterized in that, It also includes optimizing the routing algorithm in the active routing pool using a connectivity-based federated DQN algorithm, deploying a local DQN model for each enhanced twin and training it based on its own perception of environmental information; it also deploys a global DQN model, which is obtained by aggregating all local DQN models across the network, and updates the active routing pool based on the network-wide environmental information.

7. The self-evolving routing method for UAV networks based on enhanced twins according to claim 6, characterized in that, The DQN global model updates the active routing pool based on the entire network environment information. Specifically: The aggregation function of the global federated model is adjusted using connection-degree-based weights; for UAV node u i If its connectivity is deg i Then the aggregation function of the global federation model based on connectivity is: Among them, w i This indicates a local evaluation of network parameters; Based on the obtained global federation model aggregation function based on connectivity, the updated global Q-value vectors for each action are obtained. Based on the updated global Q-value vectors of each action, the routing algorithm in the active routing pool is optimized.

8. The self-evolving routing method for UAV networks based on enhanced twins according to claim 7, characterized in that, Based on the updated global Q-value vectors of each action, the routing algorithms in the active routing pool are optimized. Specifically, a mask vector is used to mask the routing algorithms already in the active routing pool. Each element in the mask vector represents a routing algorithm from the routing algorithm library, and each element has a value of 0 or 1. Correspondingly, a value of 0 means that the routing algorithm is masked, and a value of 1 means that the routing algorithm can be selected. When updating the active routing pool, the bits corresponding to the m routing algorithms already in the active routing pool in the mask vector are set to 0, and all other bits are set to 1. By performing a dot product between the mask vector and the global Q-value vector of each action, the routing algorithm in the active routing pool can be optimized.

Images