An unmanned cluster geographic location routing method based on neural network and iterative learning

Through the unmanned cluster geolocation routing method based on neural network and iterative learning, the problems of frequent topological changes and insufficient resources in the drone network are solved, and efficient routing and network stability are achieved, suitable for high-speed mobile unmanned machine environment.

CN114828146BActive Publication Date: 2024-11-26CHINA ELECTRONICS TECH GRP NO 7 RES INST
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210412303.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-19
Publication Date
2024-11-26
Estimated Expiration
2042-04-19

AI Technical Summary

Technical Problem

The existing unmanned cluster routing methods are difficult to ensure stability and reliability in drone networks facing high-speed movement and drastic changes in topology, and traditional routing protocols are difficult to operate effectively when drone resources are insufficient.

Method used

Using unmanned cluster geolocation routing method based on neural network and iterative learning, predictive rules are generated through offline training, and multiple value function sub-gradients are designed to prevent local convergence. When forming networks, neighbor parameter prediction tables are generated according to prediction rules, and the next step is selected. Repost with one jump.

Benefits of technology

This method can adapt to network environments with frequent topological changes, reduce the computing volume and power consumption of drones, adapt to environments with scarce onboard computing resources and power supplies, and improve routing efficiency and network stability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114828146B_ABST
    Figure CN114828146B_ABST
Patent Text Reader

Abstract

The present invention discloses an unmanned cluster geographic location routing method based on neural network and iterative learning, comprising the following steps: before networking, offline training is performed on historical data based on neural network and iterative learning, prediction rules are obtained through training, and multiple value function sub-gradients are designed during training to prevent local convergence; during networking, a neighbor parameter prediction table is generated according to the prediction rules; and the next hop is selected for forwarding according to the neighbor parameter prediction table and on demand. The present invention adopts offline training to reduce the amount of calculation and power consumption of the unmanned aerial vehicle, and the learning and training of historical data are all completed offline. There is no need for the unmanned cluster to perform online training during networking, which greatly reduces the amount of calculation and power consumption of the unmanned aerial vehicle, and reduces the burden on the onboard computer and power supply; it effectively avoids a large number of iterations and calculations on the aircraft, and consumes a large amount of power, thereby adapting to the environment where onboard computing resources and power supply are scarce.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of network switching technology, and more specifically, to an unmanned cluster geographic location routing method based on neural network and iterative learning. Background Art

[0002] Drones are highly flexible, can replace humans to perform tasks, reduce loss of life and property, and have the advantages of low cost. With the maturity of drone technology, drones are widely used in various fields. However, since a single drone cannot perform large-scale tasks, the concept of unmanned swarms has been proposed. The coordinated work of drones in an unmanned swarm requires communication between drones. Therefore, unmanned swarms need to be networked in the air, and routing methods suitable for unmanned swarms have become one of the research focuses.

[0003] The high-speed movement of drones causes drastic changes in topology and sparse nodes. In addition, some transmission tasks have high real-time requirements. The above characteristics and requirements have brought great challenges to traditional routing protocols. Therefore, when conducting drone aerial networking, the design and improvement of routing methods should focus on the kinematic and dynamic characteristics of drones and unmanned clusters, as well as the characteristics of unmanned cluster networks, so as to ensure the stability and reliability of the network.

[0004] Self-organizing routing protocols include static routing protocols, active routing protocols, reactive routing protocols, hybrid routing protocols, and location-based routing protocols. Although static routing protocols are widely used in unmanned cluster networks, their reliability is poor; dynamic routing protocols can use networks with frequent topology changes, but their control overhead is large and it is difficult to adapt to the narrowband network of unmanned clusters; reactive routing protocols have low overhead, but high first packet delay; hybrid routing protocols, although they solve some of the shortcomings of a priori routing protocols and reactive routing protocols, are difficult to define the scope of using a priori routing protocols; and location-based routing protocols, because they fully consider and use the location information of unmanned clusters, can not only reduce control overhead, but also reduce latency, and are more suitable for unmanned cluster networking than the other four routing protocols mentioned above.

[0005] Machine learning can learn and train historical data, summarize the laws of things and predict the future. When it is applied to unmanned cluster routing methods, it can learn and train based on the data of historical tasks to obtain the laws of networking. Therefore, compared with traditional routing protocols without machine learning, routing protocols or methods based on machine learning can predict and optimize routing.

[0006] As machine learning algorithms become more mature, many scholars have begun to use machine learning algorithms to design and improve routing methods. Machine learning algorithms widely used in routing methods include reinforcement learning, neural networks, and decision trees, which can take into account requirements such as low latency and load balancing. However, some studies are more applicable to specific scenarios such as Internet of Vehicles with small node height differences, underwater network environments with severe delays, and intertidal zones, such as the reinforcement learning-based urban scene Internet of Vehicles multicast routing method proposed by Xidian University, the machine learning-based low-latency routing method for intertidal zone sensor networks proposed by Zhejiang University, the location information-based Internet of Vehicles reinforcement learning routing method proposed by Nanjing University of Posts and Telecommunications, and the Q-learning ant colony routing method for underwater multi-agents proposed by Tsinghua University; or focus on multicast, multi-data center, software-defined network and delay-tolerant network, such as the reinforcement learning-based multi-data center energy-saving routing method and system proposed by Shandong University, the optimal path selection algorithm based on machine learning under SDN proposed by Xi'an Jiaotong University, the intelligent routing decision method based on DDPG reinforcement learning algorithm proposed by Xidian University, and the software-defined network routing method based on multi-agent reinforcement learning proposed by Nanjing University of Science and Technology; some studies have not considered the situation where various resources of the airborne system are insufficient, including power and computing. Resources, online learning routing protocols are difficult to run on drones, such as the intelligent routing method based on deep reinforcement learning technology in a wireless network environment proposed by Nanjing University of Technology, the routing optimization method and system based on graph neural network and deep reinforcement learning proposed by Huazhong University of Science and Technology, the network routing planning method and system based on BP neural network ant colony algorithm proposed by Zhejiang Gongshang University, and the Q-learning-based flying self-organizing network QoS routing method proposed by Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences. In addition, some machine learning-based routing protocols for unmanned systems do not consider the frequent movement of nodes, large movement range, or even node damage, and do not design routing protocols based on the geographical location of unmanned clusters, so the efficiency is low, such as the unmanned system network adaptive routing method and system based on deep reinforcement learning proposed by the Institute of Computing Technology of China, the mobile self-organizing network routing method based on reinforcement learning proposed by Shanghai Institute of Electric Power, and the distributed intelligent routing method for drone network slicing proposed by the University of Electronic Science and Technology of China. Summary of the invention

[0007] In order to solve the problems of the above existing technologies, the present invention provides an unmanned cluster geographic location routing method based on neural network and iterative learning.

[0008] In order to achieve the above-mentioned purpose of the present invention, the technical scheme adopted is as follows:

[0009] A method for unmanned cluster geographic location routing based on neural network and iterative learning, the method comprising the following steps:

[0010] Before networking, offline training of historical data is performed based on neural networks and iterative learning to obtain prediction rules. During training, multiple value function sub-gradients are designed, including the steepest descent sub-gradient, proportional sub-gradient, and micro-molecular gradient, to prevent local convergence.

[0011] When networking, a neighbor parameter prediction table is generated according to the prediction rules;

[0012] The next hop is selected for forwarding based on the neighbor parameter prediction table and demand.

[0013] Preferably, the historical data includes a set of m aerial mobile nodes, a set of neighbors of each node, and transmission characteristic parameters.

[0014] Furthermore, the transmission characteristic parameters include the distance between the current node and the neighboring node, the distance between the destination node and the neighboring node, the angle between the relative position vector between the neighboring node and the current node and the relative position vector between the destination node and the current node, the velocity vector of the neighboring node relative to the current node and the angle between the relative position vector between the neighboring node and the current node, the transmission bandwidth between the current node and the neighboring node, the delay between the current node and the neighboring node, the number of hops, and the transmission time; wherein, the current node and the destination node do not overlap.

[0015] Furthermore, the distance between the current node and the neighboring node, the distance between the destination node and the neighboring node, the angle between the relative position vector between the neighboring node and the current node and the relative position vector between the destination node and the current node, the velocity vector of the neighboring node relative to the current node and the angle between the relative position vector between the neighboring node and the current node, the transmission bandwidth between the current node and the neighboring node, and the time delay between the current node and the neighboring node are used as training inputs for offline training of the neural network;

[0016] The number of hops and transmission time are taken as the output pair of the input neural network.

[0017] Furthermore, the offline training of historical data based on neural network and iterative learning is carried out in the following specific steps:

[0018] S101: Select the values ​​of various constant parameters, including the number of neurons in the first hidden layer and the second hidden layer of the neural network and Coefficient κ hh and κ hy , the diagonal matrix λ y ,λ h and λ u The size of the value function standard value V req , the maximum number of iterations k max , training step length η;

[0019] S102: Initialize a first weighted matrix from input layer nodes to hidden layer nodes, a second weighted matrix from first hidden layer nodes to second hidden layer nodes, and a third weighted matrix from second hidden layer nodes to output layer nodes; wherein k represents an iteration number;

[0020] S103: for n sets of training pairs, respectively calculating the output of the first hidden layer node, the output of the second hidden layer node and the output pair matrix of the kth iteration, wherein the output pair matrix includes a hop count prediction value and a transmission time prediction value;

[0021] S104: Calculate the first value function V corresponding to the first weighting matrix, the second weighting matrix, and the third weighting matrix respectively according to the calculated output of the first hidden layer node, the output of the second hidden layer node, and the output pair matrix. uh (k), the second value function V hh (k), the third value function V hy (k);

[0022] S105: If ||V(k)|| ∞ ≤V req , where V(k) = [V uh (k) V hh (k) V hy (k)] is the value function matrix, ||·|| ∞ is an infinite norm; or the number of iterations reaches k to the set maximum number k max , then the training ends and the process goes to step S111; otherwise, the process goes to step S106;

[0023] S106: respectively calculating the cost function sub-gradients corresponding to the first weighted matrix, the second weighted matrix, and the third weighted matrix, wherein the cost function sub-gradients include a steepest descent sub-gradient, a proportional sub-gradient, and a micro-molecule gradient;

[0024] S107: Calculate the value function gradients corresponding to the first weighting matrix, the second weighting matrix, and the third weighting matrix respectively according to the calculated steepest descent sub-gradient, proportional sub-gradient, and micro-molecule gradient;

[0025] S108: Enter the next iteration, k=k+1;

[0026] S109: updating the corresponding first weighted matrix, second weighted matrix, and third weighted matrix according to the value function gradients corresponding to the first weighted matrix, the second weighted matrix, and the third weighted matrix obtained by the last iteration calculation;

[0027] S110: repeat steps S103-S105 until the iteration end condition is met, and then proceed to step S111;

[0028] S111: The updated first weighting matrix, second weighting matrix, and third weighting matrix are used as optimal weighting matrices and as prediction rules.

[0029] Furthermore, a neighbor parameter prediction table is generated according to the prediction rules, as follows:

[0030] S201: When networking, obtain the current node X c and destination node X d information;

[0031] S202: If the destination node is X d ∈χ c,B , then directly select X d is the next hop; otherwise, go to the next step;

[0032] S203: Obtain the next-hop feasible solution set x from neighbor information c,B , select neighbor node X j ∈χ B , where j∈[1,m], According to the prediction rule, the next hop is predicted to be X. j When , the number of hops and transmission time of sending a message to the destination node;

[0033] S204: traverse all neighbor nodes, repeat step S203 to predict their hop counts and transmission times, and obtain a neighbor parameter prediction table.

[0034] Furthermore, the next hop is predicted and selected based on the transmission characteristic parameters, and the next hop is predicted to be X j The number of hops to send a message to the destination node and transmission time The formula is as follows:

[0035]

[0036] In the formula, Indicates that according to the distance between the current node and the neighboring node, Indicates the distance between the destination node and its neighboring nodes. Represents the angle between the relative position vector of the neighbor node and the current node and the relative position vector of the destination node and the current node. Represents the angle between the velocity vector of the current node and the relative position vector of the neighboring node and the current node. Represents the current node and neighbor node X j The transmission bandwidth, Represents the current node and neighbor node X j delay.

[0037] Preferably, the next hop is selected for forwarding according to the neighbor parameter prediction table and on demand, and the current routing information is stored in a database; before networking, historical data is obtained from the database.

[0038] An aerial mobile device, comprising an offline training module, a prediction module, and a forwarding selection module;

[0039] Before networking, the offline training module is trained based on historical data to obtain prediction rules. During training, multiple value function sub-gradients are designed, including the steepest descent sub-gradient, proportional sub-gradient, and micro-molecular gradient, to prevent local convergence.

[0040] When networking, the prediction module generates a neighbor parameter prediction table based on the prediction rules;

[0041] The forwarding selection module selects the next hop for forwarding based on the neighbor parameter prediction table and demand, and stores the current routing information in the database.

[0042] A computer system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of an unmanned cluster geographic location routing method based on a neural network and iterative learning are implemented.

[0043] The beneficial effects of the present invention are as follows:

[0044] 1. The present invention is aimed at unmanned swarm aerial networking, combines neural networks and iterative learning to design a routing method based on geographic location, fully utilizes the navigation function and movement characteristics of unmanned swarms, and adapts to confrontational environments with frequent changes in network topology.

[0045] 2. In addition, the present invention adopts offline training to reduce the computing power and power consumption of the UAV. The learning and training of historical data are completed offline. There is no need for the unmanned cluster to perform online training when networking, which greatly reduces the computing power and power consumption of the UAV, and reduces the burden on the onboard computer and power supply; effectively avoids a large number of iterations and calculations on the aircraft and consumes a large amount of power, thereby adapting to the environment where onboard computing resources and power are scarce. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flow chart of the unmanned cluster geographic location routing method based on neural network and iterative learning of the present invention.

[0047] Figure 2 It is a detailed step flow chart of the method described in the present invention.

[0048] Figure 3 It is a schematic diagram of the physical meaning of data between the current node, neighboring nodes, and destination node of the present invention.

[0049] Figure 4 It is a workflow diagram of the offline training of the present invention.

[0050] Figure 5 It is a schematic diagram of the structure and training process of the neural network of the present invention.

[0051] Figure 6 It is a historical data schematic diagram of the present invention.

[0052] Figure 7 It is a workflow diagram of the present invention for generating a neighbor parameter prediction table and forwarding selection.

[0053] Figure 8 This is an example diagram of neighbor node information of the present invention.

[0054] Fig. 9 yes Figure 7 Transmission parameter characteristic information of the intermediate nodes.

[0055] Fig.10 This is an example diagram of the neighbor parameter prediction table of the present invention.

[0056] Fig.11 It is a system principle diagram of the aerial mobile equipment described in the present invention. DETAILED DESCRIPTION

[0057] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0058] Example 1

[0059] Taking into account that the position of each drone node in an unmanned cluster changes frequently but regularly, and the regularity is related to the tasks performed by the unmanned cluster and the kinematics and dynamics of the unmanned cluster, the present invention combines neural networks and iterative learning to design a location-based routing method, which can adapt to network environments with frequent topology changes, and adopts an offline training method to avoid a large amount of calculation and storage on the airborne computer, resulting in high power consumption, thereby adapting to the airborne environment. The aerial mobile nodes involved in this embodiment include but are not limited to drone nodes, balloon nodes, glider nodes, airship nodes, airplane nodes, and helicopter nodes. This embodiment is described in detail using drone nodes as an example as follows:

[0060] like Figure 1 , Figure 2 As shown, an unmanned cluster geographic location routing method based on neural network and iterative learning, the method comprising the following steps:

[0061] Before networking, offline training of historical data is performed based on neural networks and iterative learning to obtain prediction rules. During training, multiple value function sub-gradients are designed, including the steepest descent sub-gradient, proportional sub-gradient, and micro-molecular gradient, to prevent local convergence.

[0062] When networking, a neighbor parameter prediction table is generated based on prediction rules without knowing the entire network topology;

[0063] The next hop is selected for forwarding based on the neighbor parameter prediction table and demand.

[0064] In a specific embodiment, the historical data includes The set of drone nodes χ={X1,X2,…,X m}、Each node X i ∈χ's neighbor set And transmission characteristic parameters. Figure 3 As shown, the transmission characteristic parameters include the transmission characteristic parameters from the current node X c ∈χ through the next hop X b ∈χ i,B Towards the destination node X d ∈χWhen sending information, the distance between the current node and the neighboring node The distance between the destination node and its neighbor nodes The relative position vector of the neighbor node and the current node and the relative position vector of the destination node and the current node Angle The velocity vector of the neighbor node relative to the current node The angle between the relative position vector of the neighbor node and the current node The transmission bandwidth between the current node and the neighboring nodes The delay between the current node and the neighboring node Hop Count Transfer time Among them, c≠d means that the current node and the destination node do not overlap; that is, a set of data is:

[0065]

[0066] Among them, n is the data sequence number.

[0067] The velocity vector of the neighbor node relative to the current node The angle between the relative position vector of the neighbor node and the current node It can be calculated according to the following formula:

[0068]

[0069] in, is the position vector of the destination node relative to the neighboring nodes, and and All obtained by navigation.

[0070] In a specific embodiment, the distance between the current node and the neighboring node is The distance between the destination node and its neighbor nodes The relative position vector of the neighbor node and the current node and the relative position vector of the destination node and the current node Angle The velocity vector of the neighbor node relative to the current node The angle between the relative position vector of the neighbor node and the current node The transmission bandwidth between the current node and the neighboring nodes The delay between the current node and the neighboring node As a training pair, input the neural network for offline training;

[0071] The number of hops Transfer time As input to the output of the neural network.

[0072] Coming soon As training, the input neural network is trained offline;

[0073] Will As input to the output of the neural network.

[0074] In this embodiment, the training pair is input to the neural network for offline training to obtain output pairs.

[0075] This embodiment first adopts an offline training method, which can avoid performing a large amount of calculation and storage on the onboard computer and generating large power consumption, thereby adapting to the onboard environment.

[0076] In a specific embodiment, the neural network described in this embodiment includes an input layer, a first hidden layer, a second hidden layer, and an output layer in sequence.

[0077] like Figure 4 , Figure 5 , Figure 6 As shown, the offline training of historical data based on neural network and iterative learning is carried out, and the specific steps are as follows:

[0078] S101: Select the values ​​of various constant parameters, including the number of neurons in the first hidden layer and the second hidden layer of the neural network and Coefficient κ hh and κ hy , the diagonal matrix λ y ,λ h and λ u The size of the value function standard value V req , the maximum number of iterations k max , training step length η; where k represents the iteration number;

[0079] S102: Initialize offline training parameters. Set the first weight matrix from input layer nodes to hidden layer nodes as The second weight matrix from the first hidden layer node to the second hidden layer node is The third weight matrix from the second hidden layer node to the output layer node is Where k represents the iteration number, the first weighting matrix, the second weighting matrix, and the third weighting matrix are initialized to W uh (0), W hh (0), W hy (0) and k=0.

[0080] S103: For n sets of training pairs, calculate the output of the first hidden layer node of the kth iteration respectively The output of the second hidden layer node And the output pair matrix The output pair matrix includes the hop count prediction value and transmission time prediction

[0081] A specific calculation method provided in this embodiment is as follows:

[0082]

[0083] in, is the training pair matrix;

[0084] S104: Output of the first hidden layer node obtained by calculation The output of the second hidden layer node And the output pair matrix Calculate the first weight matrix separately The second weighting matrix The third weighting matrix The corresponding first value function V uh (k), the second value function V hh (k), the third value function V hy (k);

[0085] A specific calculation formula provided in this embodiment is as follows:

[0086]

[0087] Among them, e y (n, k) is the error of the nth (n≤N) training set for the output node, e1(n, k) is the virtual error of the nth training set for the first hidden layer node, e2(n, k) is the virtual error of the nth training set for the second hidden layer node, κ hh ≥0 and κhy ≥0 is a constant parameter, and ||·||2 is the second norm.

[0088]

[0089] in, is the theoretical value of the output node of the nth training pair, z 2,T (n, k) is the virtual theoretical value of the second hidden layer node, z 1,T (n, k) is the virtual theoretical value of the first hidden layer node, λ y ,λ h and λ u is a positive definite diagonal matrix.

[0090] S105: If the value function V uh (k), V hh (k) and V hy (k) satisfies the requirement, that is, if ||V(k)|| ∞ ≤V req , where V(k) = [V uh (k) V hh (k) V hy (k)] is the value function matrix, ||·|| ∞ is the infinite norm, V req is the standard value of the value function; or the number of iterations reaches k to the set maximum number k max , then the training ends and the process goes to step S111; otherwise, the process goes to step S106;

[0091] S106: Calculate the first weight matrix W respectively hy (k), the second weighting matrix W hh (k), the third weighting matrix W uh (k) The corresponding value function sub-gradient, wherein the value function sub-gradient includes the steepest descent sub-gradient, the proportional sub-gradient and the micro-molecule gradient; Specifically, the first weighting matrix W hy The sub-gradients of the value function corresponding to (k) are respectively expressed as the steepest descent sub-gradients Proportional sub-gradient and micromolecular gradient The second weight matrix W hh The sub-gradients of the value function corresponding to (k) are respectively expressed as the steepest descent sub-gradients Proportional sub-gradient and micromolecular gradient The third weight matrix W uh The sub-gradients of the value function corresponding to (k) are respectively expressed as the steepest descent sub-gradients Proportional sub-gradient and micromolecular gradient

[0092] in, and Calculated according to the following formula:

[0093]

[0094]

[0095]

[0096] in, For the matrix The element in the i-th row and j-th column, (·)' is the derivative, W hy Virtual theoretical value of (k) With W hy (k), E(·) is the expectation.

[0097] and Calculated according to the following formula:

[0098]

[0099]

[0100]

[0101] in, For the matrix The element in row i and column j, W hh Virtual theoretical value of (k) With W hh (k) difference.

[0102] and Calculated according to the following formula:

[0103]

[0104]

[0105]

[0106] in, For the matrix The element in row i and column j, W uh Virtual theoretical value of (k) With W hh (k) difference.

[0107] In this embodiment, the positive and negative signs of the differences between the virtual theoretical values ​​and the iterative values ​​of each weighting matrix are calculated according to the following formula:

[0108]

[0109] S107: According to the calculated steepest descent sub-gradient, proportional sub-gradient and micro-molecule gradient, the cost function gradients corresponding to the first weighted matrix, the second weighted matrix and the third weighted matrix are calculated respectively; each cost function sub-gradient is used to prevent local convergence of each weighted matrix, and the specific calculation formula is as follows:

[0110]

[0111] Among them, G hy (k) represents the value function gradient corresponding to the first weighted matrix, G hh (k) represents the value function gradient corresponding to the second weighted matrix, G uh (k) represents the value function gradient corresponding to the third weighting matrix.

[0112] S108: Enter the next iteration, k=k+1;

[0113] S109: According to the value function gradients corresponding to the first weighted matrix, the second weighted matrix, and the third weighted matrix calculated in the previous iteration, the corresponding first weighted matrix, the second weighted matrix, and the third weighted matrix are updated; specifically, according to the value function gradient G uh (k-1), G hh (k-1) and G hy (k-1) Update the weight matrix to W uh (k), W hh (k) and W hy (k):

[0114]

[0115] Among them, the training step length η>0 is a constant parameter.

[0116] S110: repeat steps S103-S105 until the iteration end condition is met, and then proceed to step S111;

[0117] S111: Obtain the updated first weighting matrix The second weighting matrix The third weighting matrix as the optimal weighting matrix and as the prediction rule.

[0118] In a specific embodiment, Figure 7 As shown, a neighbor parameter prediction table is generated according to the prediction rules, as follows:

[0119] S201: When networking, obtain the current node X c and destination node X d information;

[0120] S202: If the destination node is X d ∈χ c,B , then directly select X d is the next hop; otherwise, go to the next step;

[0121] S203: Obtain the next-hop feasible solution set x from neighbor information c,B , select neighbor node X j ∈χ B , where j∈[1,m], The next hop is predicted and selected based on the transmission characteristic parameters. The next hop is predicted to be X. j When , the number of hops and transmission time of sending a message to the destination node;

[0122] S204: traverse all neighbor nodes, repeat step S203 to predict their hop counts and transmission times, and obtain a neighbor parameter prediction table.

[0123] In this embodiment, if step S202 determines that the destination node is a neighbor node X d ∈χ c,B , then the routing information is stored in the database;

[0124] In a specific embodiment, the next hop is selected for forwarding according to the neighbor parameter prediction table and on demand, and the current routing information is stored in a database; before networking, historical data is obtained from the database.

[0125] The neighbor parameter prediction table obtained in this embodiment is as follows:

[0126]

[0127] In a specific embodiment, the next hop is selected based on the transmission characteristic parameter prediction, and the next hop is predicted to be X. j The number of hops to send a message to the destination node and transmission time The formula is as follows:

[0128]

[0129] In the formula, Indicates that according to the distance between the current node and the neighboring node, Indicates the distance between the destination node and its neighboring nodes. Represents the angle between the relative position vector of the neighbor node and the current node and the relative position vector of the destination node and the current node. Represents the angle between the velocity vector of the current node and the relative position vector of the neighboring node and the current node. Represents the current node and neighbor node X j The transmission bandwidth, Represents the current node and neighbor node X j delay.

[0130] like Figure 8 , Fig. 9 As shown in the figure, it is also assumed that the unmanned cluster includes 6 aerial mobile nodes. When the offline training is completed and the aerial network is formed, it is assumed that the current node is node 2, that is, X c =X2, the destination node is node 6, that is, X d =X6, the neighbors of node 2 are nodes 3, 4 and 5, that is, χ B ={X3,X4,X5}. When the prediction module is working, it obtains the relevant information of the above nodes, including: the distance of node 3 relative to node 2 The distance of node 6 relative to node 3 The angle between the position vector of node 3 relative to node 2 and the position vector of node 6 relative to node 2 The angle between the velocity vector of node 3 relative to node 2 and the position vector of node 6 relative to node 2 Bandwidth of nodes 2 and 3 Latency between nodes 2 and 3 The distance of node 4 relative to node 2 The distance of node 6 relative to node 4 The angle between the position vector of node 4 relative to node 2 and the position vector of node 6 relative to node 2 The angle between the velocity vector of node 4 relative to node 2 and the position vector of node 6 relative to node 2 Bandwidth of nodes 2 and 4 Latency between nodes 2 and 4 The distance of node 5 relative to node 2 The distance of node 5 relative to node 6 The angle between the position vector of node 5 relative to node 2 and the position vector of node 6 relative to node 2 The angle between the velocity vector of node 5 relative to node 2 and the position vector of node 6 relative to node 2 Bandwidth of nodes 2 and 5 Latency between nodes 2 and 5 The optimal weighted matrix obtained from offline training is used as the prediction rule for prediction, and the neighbor parameter prediction table is obtained, such as Fig.10 As shown, including the predicted value of the number of hops from node 2 to node 6 when node 3 is selected as the next hop and transmission time prediction When node 4 is selected as the next hop, the predicted value of the number of hops from node 2 to node 6 and transmission time prediction When node 5 is selected as the next hop, the predicted value of the number of hops from node 2 to node 6 and transmission time prediction The forwarding selection module can select the next hop based on the neighbor parameter prediction table according to the criterion of small hop number or small transmission time. For example, when the requirement is small hop number, the next hop selected is When the requirement is short transmission time, the next hop selected is Example 2

[0131] like Fig.11 As shown, an aerial mobile device includes an offline training module, a prediction module, and a forwarding selection module;

[0132] Before networking, the offline training module is trained based on historical data to obtain prediction rules. During training, multiple value function sub-gradients are designed, including the steepest descent sub-gradient, proportional sub-gradient, and micro-molecular gradient, to prevent local convergence.

[0133] When networking, the prediction module generates a neighbor parameter prediction table based on the prediction rules;

[0134] The forwarding selection module selects the next hop for forwarding according to the neighbor parameter prediction table and demand, and stores the current routing information in the database; before networking, historical data is obtained from the database.

[0135] The offline training module performs offline training on historical data based on a neural network and iterative learning, thereby implementing the method of steps S101 to S111 in Example 1.

[0136] The forwarding selection module generates a neighbor parameter prediction table according to the prediction rule, and implements the method of steps S201 to S204 in Example 1.

[0137] The aerial mobile device described in this embodiment refers to a flying device manufactured by humans, capable of taking off from the ground, flying in space and controlled by humans, flying within the atmosphere or outside the atmosphere (space), including but not limited to drones, balloons, gliders, airships, airplanes, and helicopters.

[0138] Example 3

[0139] A computer system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following method steps are implemented:

[0140] Before networking, offline training of historical data is performed based on neural networks and iterative learning to obtain prediction rules. During training, multiple value function sub-gradients are designed, including the steepest descent sub-gradient, proportional sub-gradient, and micro-molecular gradient, to prevent local convergence.

[0141] When networking, a neighbor parameter prediction table is generated according to the prediction rules;

[0142] The next hop is selected for forwarding based on the neighbor parameter prediction table and demand.

[0143] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor.

[0144] Obviously, the above embodiments of the present invention are only examples for clearly explaining the present invention, and are not intended to limit the implementation methods of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. An unmanned cluster geographic location routing method based on neural network and iterative learning, characterized by: The method comprises the following steps: Before networking, offline training of historical data is performed based on neural networks and iterative learning to obtain prediction rules. During training, multiple value function sub-gradients are designed, including the steepest descent sub-gradient, proportional sub-gradient, and micro-molecular gradient, to prevent local convergence. When networking, a neighbor parameter prediction table is generated according to the prediction rules; Select the next hop for forwarding based on the neighbor parameter prediction table and demand; The offline training of historical data based on neural network and iterative learning is carried out in the following specific steps: S101: Select the values ​​of various constant parameters, including the number of neurons in the first hidden layer and the second hidden layer of the neural network and Coefficient κ hh and κ hy , the diagonal matrix λ y ,λ h and λ u The size of the value function standard value V req , the maximum number of iterations k max , training step length η; where k represents the iteration number; S102: Initialize a first weighted matrix from input layer nodes to hidden layer nodes, a second weighted matrix from first hidden layer nodes to second hidden layer nodes, and a third weighted matrix from second hidden layer nodes to output layer nodes; S103: for n sets of training pairs, respectively calculating the output of the first hidden layer node, the output of the second hidden layer node and the output pair matrix of the kth iteration, wherein the output pair matrix includes a hop count prediction value and a transmission time prediction value; S104: Calculate the first value function V corresponding to the first weighting matrix, the second weighting matrix, and the third weighting matrix respectively according to the calculated output of the first hidden layer node, the output of the second hidden layer node, and the output pair matrix. uh (k), the second value function V hh (k), the third value function V hy (k); S105: If ||V(k)|| ∞ ≤V req , where V(k) = [V uh (k) V hh (k) V hy (k)] is the value function matrix, ||·|| ∞ is an infinite norm; or the number of iterations reaches k to the set maximum number k max , then the training ends and the process goes to step S111; otherwise, the process goes to step S106; S106: respectively calculating the cost function sub-gradients corresponding to the first weighted matrix, the second weighted matrix, and the third weighted matrix, wherein the cost function sub-gradients include a steepest descent sub-gradient, a proportional sub-gradient, and a micro-molecule gradient; S107: Calculate the value function gradients corresponding to the first weighting matrix, the second weighting matrix, and the third weighting matrix respectively according to the calculated steepest descent sub-gradient, proportional sub-gradient, and micro-molecule gradient; S108: Enter the next iteration, k=k+1; S109: updating the corresponding first weighted matrix, second weighted matrix, and third weighted matrix according to the value function gradients corresponding to the first weighted matrix, the second weighted matrix, and the third weighted matrix obtained by the last iteration calculation; S110: repeat steps S103-S105 until the iteration end condition is met, and then proceed to step S111; S111: The updated first weighting matrix, second weighting matrix, and third weighting matrix are used as optimal weighting matrices and as prediction rules.

2. The unmanned cluster geographic location routing method based on neural network and iterative learning according to claim 1, characterized in that: The historical data includes a set of m aerial mobile nodes, a set of neighbors of each node, and transmission characteristic parameters.

3. The unmanned cluster geographic location routing method based on neural network and iterative learning according to claim 2, characterized in that: The transmission characteristic parameters include the distance between the current node and the neighboring node, the distance between the destination node and the neighboring node, the angle between the relative position vector between the neighboring node and the current node and the relative position vector between the destination node and the current node, the velocity vector of the neighboring node relative to the current node and the angle between the relative position vector between the neighboring node and the current node, the transmission bandwidth between the current node and the neighboring node, the delay between the current node and the neighboring node, the number of hops, and the transmission time; wherein the current node and the destination node do not overlap.

4. The unmanned cluster geographic location routing method based on neural network and iterative learning according to claim 3 is characterized by: The distance between the current node and the neighboring node, the distance between the destination node and the neighboring node, the angle between the relative position vector between the neighboring node and the current node and the relative position vector between the destination node and the current node, the velocity vector of the neighboring node relative to the current node and the angle between the relative position vector between the neighboring node and the current node, the transmission bandwidth between the current node and the neighboring node, and the time delay between the current node and the neighboring node are used as training inputs to perform offline training on the neural network; The number of hops and transmission time are taken as the output pair of the input neural network.

5. The unmanned cluster geographic location routing method based on neural network and iterative learning according to claim 4, characterized in that: Generate a neighbor parameter prediction table based on the prediction rules, as follows: S201: When networking, obtain the current node X c and destination node X d information; S202: If the destination node is Then select X directly d is the next hop; otherwise, go to the next step; S203: Obtain the next-hop feasible solution set from neighbor information Select neighbor nodes where j∈[1,m], According to the prediction rule, the next hop is predicted to be X. j When , the number of hops and transmission time of sending a message to the destination node; S204: traverse all neighbor nodes, repeat step S203 to predict their hop counts and transmission times, and obtain a neighbor parameter prediction table.

6. The unmanned cluster geographic location routing method based on neural network and iterative learning according to claim 5, characterized in that: The next hop is predicted and selected based on the transmission characteristic parameters. The predicted next hop is X j The number of hops to send a message to the destination node and transmission time The formula is as follows: In the formula, Indicates that according to the distance between the current node and the neighboring node, Indicates the distance between the destination node and its neighboring nodes. Represents the angle between the relative position vector of the neighbor node and the current node and the relative position vector of the destination node and the current node. Represents the angle between the velocity vector of the current node and the relative position vector of the neighboring node and the current node. Represents the current node and neighbor node X j The transmission bandwidth, Represents the current node and neighbor node X j The delay of W uh Represents the first weight matrix from the input layer nodes to the hidden layer nodes; W hh represents the second weighting matrix from the first hidden layer node to the second hidden layer node; W hy Represents the third weighting matrix from the second hidden layer nodes to the output layer nodes.

7. The unmanned cluster geographic location routing method based on neural network and iterative learning according to claim 1, characterized in that: The next hop is selected for forwarding based on the neighbor parameter prediction table and demand, and the current routing information is stored in the database. Before networking, historical data is obtained from the database.

8. An aerial mobile device, characterized in that: Includes offline training module, prediction module, and forwarding selection module; Before networking, the offline training module is trained based on historical data to obtain prediction rules. During training, multiple value function sub-gradients are designed, including the steepest descent sub-gradient, proportional sub-gradient, and micro-molecular gradient, to prevent local convergence. When networking, the prediction module generates a neighbor parameter prediction table based on the prediction rules; The forwarding selection module selects the next hop for forwarding based on the neighbor parameter prediction table and demand, and stores the current routing information in the database; The specific steps of training based on historical data are as follows: S101: Select the values ​​of various constant parameters, including the number of neurons in the first hidden layer and the second hidden layer of the neural network and Coefficient κ hh and κ hy , the diagonal matrix λ y ,λ h and λ u The size of the value function standard value V req , the maximum number of iterations k max , training step length η; where k represents the iteration number; S102: Initialize a first weighted matrix from input layer nodes to hidden layer nodes, a second weighted matrix from first hidden layer nodes to second hidden layer nodes, and a third weighted matrix from second hidden layer nodes to output layer nodes; S103: for n sets of training pairs, respectively calculating the output of the first hidden layer node, the output of the second hidden layer node and the output pair matrix of the kth iteration, wherein the output pair matrix includes a hop count prediction value and a transmission time prediction value; S104: Calculate the first value function V corresponding to the first weighting matrix, the second weighting matrix, and the third weighting matrix respectively according to the calculated output of the first hidden layer node, the output of the second hidden layer node, and the output pair matrix. uh (k), the second value function V hh (k), the third value function V hy (k); S105: If ||V(k)|| ∞ ≤V req , where V(k) = [V uh (k) V hh (k) V hy (k)] is the value function matrix, ||·|| ∞ is an infinite norm; or the number of iterations reaches k to the set maximum number k max , then the training ends and the process goes to step S111; otherwise, the process goes to step S106; S106: respectively calculating the cost function sub-gradients corresponding to the first weighted matrix, the second weighted matrix, and the third weighted matrix, wherein the cost function sub-gradients include a steepest descent sub-gradient, a proportional sub-gradient, and a micro-molecule gradient; S107: Calculate the value function gradients corresponding to the first weighting matrix, the second weighting matrix, and the third weighting matrix respectively according to the calculated steepest descent sub-gradient, proportional sub-gradient, and micro-molecule gradient; S108: Enter the next iteration, k=k+1; S109: updating the corresponding first weighted matrix, second weighted matrix, and third weighted matrix according to the value function gradients corresponding to the first weighted matrix, the second weighted matrix, and the third weighted matrix obtained by the last iteration calculation; S110: repeat steps S103-S105 until the iteration end condition is met, and then proceed to step S111; S111: The updated first weighting matrix, second weighting matrix, and third weighting matrix are used as optimal weighting matrices and as prediction rules.

9. A computer system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.