An incremental network-based UAV learning method with leader guidance

Abstract

The present invention relates to the field of intelligent collaboration and reinforcement learning of unmanned aerial vehicles, and discloses a learning method for unmanned aerial vehicles with the guidance of a leader based on an incremental network, including: establishing an iterative model of unmanned aerial vehicles based on different environments, establishing a profit matrix; establishing an incremental network generation algorithm; establishing a dynamic model that takes into account the influence of multiple nonlinear environments on unmanned aerial vehicles; and introducing an incremental network with the guidance of a leader to cope with environmental influences. The incremental network generation algorithm proposed in the present invention generates an effective communication network based on the information convergence between unmanned aerial vehicles to improve the learning efficiency of unmanned aerial vehicles; the unmanned aerial vehicle dynamics model based on the continuous action iteration dilemma has multiple nonlinear layers to simulate the different environmental influences on the unmanned aerial vehicle. In addition, based on the incremental network and the dynamics model of the unmanned aerial vehicle, the present invention uses the Lyapunov function to prove the convergence of the proposed model, and mathematically proves that the proposed model is stable.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle (UAV) intelligent collaboration and reinforcement learning, and in particular to an incremental network-based UAV learning method with leader guidance. Background Art

[0002] Peer learning refers to the process of mutual learning, knowledge sharing, and experience exchange among peers. This learning method emphasizes collaboration, interaction, and communication among students, rather than relying on the traditional one-way teacher-student relationship for classroom teaching. In this learning method, each student can play to his or her strengths, benefit from the experience and knowledge of others, and deepen his or her understanding of the learning content by teaching others. Peer learning has been widely used in the field of education, especially in higher education. It can promote communication and interaction among students, enhance students' autonomous learning and critical thinking abilities, and improve students' learning efficiency and grades. In addition to its application in schools, peer learning can also be applied to various fields such as drone cluster control to promote collective learning and knowledge sharing. However, it should be noted that the interactive relationship between students will also affect the effectiveness of peer learning, because some groups may not be able to achieve information convergence or reach consensus due to ineffective communication. In addition, each student is a highly unique individual, and using a single mathematical model to simulate their behavior will result in limited applicability of the research results. Connectivity is an important influencing factor in peer learning research. In previous studies, the efficiency of node utilization was maximized by combining a shared wireless channel network with peer learning. There are also studies that investigate the propagation performance of the network in peer learning. It should be noted that in these studies, the communication relationship between students is static. But in actual classrooms, dynamic communication relationships are more conducive to improving learning efficiency and enhancing classroom cohesion. Accurate mathematical modeling of students is an important basic research for peer learning with the guidance of a leader. In previous studies, some scholars proposed a method to improve student performance based on survey data analysis, and some scholars studied methods to model student behavior under noise conditions. It can be found that at present, in the process of mathematical modeling, all student models are consistent, which does not conform to the actual situation, because each student in reality is very unique. At the same time, the learning effect in the classroom is not only affected by the leader, but also by various aspects such as the environment. Therefore, how to simulate the influence of the environment with mathematical methods is also a problem that needs to be considered. In addition, each drone is a unique individual, and using a single mathematical model to imitate their behavior will result in limited applicability of the research results.

[0003] Effective communication depends on the degree of information convergence between students. Students' cooperative behavior is affected by factors such as personnel and project culture, discipline and institutional background, as well as curriculum, regulations and funding restrictions. In the process of mathematical modeling, it can be observed that the model results are greatly affected by parameters. Different parameters may lead to completely different results, which reduces the applicability of the model. Therefore, proving that the convergence of the model is not affected by parameters can make it more applicable. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the object of the present invention is to provide a UAV learning method with leader guidance based on incremental network.

[0005] To achieve the above objectives, the present invention provides a UAV learning method with the guidance of a leader based on an incremental network, which adopts the following technical solutions:

[0006] A method for learning an unmanned aerial vehicle with the guidance of a leader based on an incremental network comprises the following steps:

[0007] Step 1: Establish an iterative model of drones based on different environments and establish a profit matrix;

[0008] Step 2: Establish an incremental network generation algorithm;

[0009] Step 3: Establish a dynamic model that considers the UAV under the influence of multiple nonlinear environments;

[0010] Step 4: Introduce an incremental network with long-term guidance to deal with environmental impacts.

[0011] Furthermore, the step 1 comprises the following steps:

[0012] Step 1.1 Establish a UAV dynamics model based on CAID;

[0013] Set x i ∈[0,1] corresponds to the strategy of drone i, where i∈{1,2,…,N} represents a drone, where x=0 represents betrayal and x=1 represents full cooperation;

[0014] Step 1.2 Establish the model profit matrix;

[0015] The payoff matrix is:

[0016]

[0017] Where μ0, μ1, μ2, and μ3 represent the benefits of each participant in the traditional evolutionary dynamics model;

[0018] Step 1.3 Establish the model fitness function

[0019] For the strategy selected by drone i, the fitness function F(x i )for:

[0020]

[0021] where a ij is an element in the adjacency matrix, x i ∈[0,1] corresponds to the strategy of drone i, x j The strategy for drone j

[0022] Furthermore, in step 2, for the non-connected undirected graph G, the Tarjan SCC algorithm is used to split it into several strongly connected components; for each pair of strongly connected components [(S i ,S j )], if S i If it cannot be reached, then j Randomly select at least one node from S i Randomly select a node in and add directed edges between them, repeating this step until a strongly connected graph is generated.

[0023] Furthermore, the pseudo code of the incremental network generation algorithm is as follows:

[0024]

[0025] Furthermore, in step 3, the dynamic model under the influence of multiple nonlinear environments can be expressed as:

[0026]

[0027] p ij is the probability of switching to the neighboring strategy, f i is the unknown nonlinear environmental impact on UAV i, f i =[sinx i ,cosx i ,…] is composed of multiple nonlinear functions, deg(v i ) is the degree of the undirected graph.

[0028] Furthermore, in step 4, a dynamic weighted incremental network with a long-machine guidance is introduced to cope with environmental influences, and the dynamic model can be expressed as:

[0029]

[0030] η i The effect is due to the best guidance from the leader.

[0031] Furthermore, the radial basis function neural network is used to fit the dynamic model of the incremental network with long-term guidance to cope with environmental influences, as follows:

[0032]

[0033] The input vector is: X = [x1,…,x N ], weight vector: W = [W1,…,W N ], is the Gaussian function, ζ is the fitting error of the nonlinear function;

[0034] So choose a suitable approximation: is the estimated value of the weight matrix, then the estimation error is

[0035] The estimated error of the function can be expressed as:

[0036] Another aspect of the present invention is to perform model convergence analysis, wherein the lemma is introduced during the analysis:

[0037] Lemma 3: If graph G k is strongly connected, then it has a spanning tree, and L k The rank is N-1

[0038] Lemma 4: If the rank of L is N-1, then 1α is the only vector in the null space of L that, for some constant α, has a steady-state x s =1α.

[0039] Furthermore, the convergence analysis method of the CAID model in the incremental network obtained in step 3 is as follows:

[0040] Assume the Lyapunov function is:

[0041]

[0042] where e i is the drone status x i The error from the equilibrium point α;

[0043] The derivative of V is

[0044]

[0045] Depend on It can be further rewritten as:

[0046]

[0047] The proof is that the CAID dynamics model with limited learning ability is convergent, where

[0048] Furthermore, the convergence analysis method of the CAID model with multiple unknown nonlinear environmental influences obtained in step 4 is as follows:

[0049] Assumptions:

[0050] 1) The unknown ideal NN weight matrix W is represented by ||W|| F ≤W M Boundary

[0051] 2) NN activation function Bounded, for the entire network:

[0052] Introduction Theorem 1: Consider the system given in Assumption 1), so that the communication graph is strongly connected, the optimal guidance level for selecting the leader is:

[0053] in is a Gaussian function

[0054] At the same time, the weight matrix update law is

[0055] w ij is the element in the unknown weight matrix NN;

[0056] Then the system: convergence

[0057] where p β ij =η i p ij , η i The best level with the guidance of the captain, p ij is the probability of switching to the neighboring strategy, f i is the unknown nonlinear environmental impact on UAV i.

[0058] prove:

[0059] Let the Lyapunov function be The derivative of the function is

[0060] is the estimation error of UAV i;

[0061] Make the following replacements:

[0062] where p β ij =η i p ij , η iThe best level with the guidance of the captain, p ij is the probability of switching to the neighboring strategy, f i is the unknown nonlinear environmental impact on UAV i;

[0063] can be written as:

[0064] When satisfied: Can get

[0065]

[0066] therefore:

[0067]

[0068] Substitute:

[0069]

[0070] If satisfied: have:

[0071]

[0072] Corresponding:

[0073] definition:

[0074] You can get:

[0075] Based on Assumption 1, let φ β max =max(φ ij β ,φ ij β )

[0076] You can get:

[0077]

[0078] At this point, the proof is complete, and the CAID dynamics model in the incremental network with long-machine guidance is convergent.

[0079] Compared with the prior art, the present invention has the following beneficial effects:

[0080] (1) The UAV learning method with continuous action iteration dilemma guided by a leader based on incremental network in the present invention, compared with the static communication relationship between UAVs in traditional UAV learning (which is inconsistent with the actual situation and also affects the effectiveness of learning), and the use of a single mathematical model to imitate their behavior (ignoring that each UAV is a unique individual), the present invention proposes an incremental network generation algorithm, which generates an effective communication network based on the information convergence between UAVs to improve the efficiency of the class;

[0081] (2) The method of the present invention introduces an incremental network with leader guidance to improve the ability of UAVs to cope with environmental interference, which can better realize the coordination and information exchange between UAVs and better complete their own tasks. After introducing the UAV learning method with continuous action iterative dilemma with leader guidance based on incremental network, a UAV dynamics model based on continuous action iterative dilemma (CAID) is designed. The model has multiple nonlinear layers to simulate the different environmental influences on UAVs. The coordination and communication of UAVs in the UAV cluster are more reliable, the level of autonomous decision-making of UAVs is optimized, and the task of cooperative control of UAVs can be better completed. In the engineering field, it can help UAVs perceive the behavior of other UAVs and the impact of the environment on themselves, so as to better realize the coordination and control between UAVs.

[0082] (3) The present invention uses the Lyapunov function to prove the convergence of the proposed model and mathematically proves that the proposed model is stable. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0084] Figure 1 This is a flow chart of the UAV learning method with leader guidance based on incremental network in Example 1;

[0085] Figure 2 It is a CAID model under the incremental network with long-term guidance;

[0086] Figure 3 is an unconnected network with N being 20;

[0087] Figure 4 is a strongly connected network with N=20;

[0088] Figure 5 Iteration results for the CAIPD model in an unconnected network;

[0089] Figure 6 Iterate the results for the CAIPD model in a strongly connected network;

[0090] Figure 7 Iteration results for the CAISD model in an unconnected network;

[0091] Figure 8 Iterate the results for the CAISD model in a strongly connected network;

[0092] Fig. 9 When N=20, the iteration of the CAIPD model in the incremental network without the guidance of the long machine;

[0093] Fig.10 When N=20, the iteration of the CAIPD model in the incremental network with the guidance of the long machine;

[0094] Fig.11 When N=20, the iteration of the CAISD model in the incremental network without the guidance of the long machine;

[0095] Fig.12 When N=20, the iteration of the CAISD model under the incremental network with the guidance of the leader. DETAILED DESCRIPTION

[0096] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0097] Example 1

[0098] A method for learning an unmanned aerial vehicle with the guidance of a leader based on an incremental network comprises the following steps:

[0099] Step 1: Establish an iterative model of drones based on different environments and establish a profit matrix;

[0100] Step 1.1 Establish a UAV dynamics model based on CAID;

[0101] The connectivity between drones can be represented by an undirected graph G = (V, A), where V = {v1, v2…, v N} is the set of drones, A∈R is the adjacency matrix, if drone v j With dronev i There is a connectivity relationship between them, and we can get: a ij =1 or a ij=0. In this way, the information transmission between drones is abstracted into a game model. In the traditional model, drones can only choose two strategies: share information or not share information. Considering various unknown nonlinear environmental influences, this paper designs a drone dynamics model based on continuous action iterative dilemma (CAID), which has multiple nonlinear layers to simulate the environmental influences on drones. Set x i ∈[0,1] corresponds to the strategy of drone i, where i∈{1,2,…,N} represents a drone, where x=0 represents betrayal and x=1 represents full cooperation.

[0102] Step 1.2 Establish the model profit matrix;

[0103] An important factor affecting evolutionary dynamics is the payoff matrix. The payoff matrix is ​​the basis for drone selection strategies, and different choices bring different returns. The payoff matrix selected in this application is:

[0104]

[0105] Where μ0, μ1, μ2, and μ3 represent the benefits of each participant in the traditional evolutionary dynamics model.

[0106] Step 1.3 Establish the model fitness function.

[0107] For the strategy selected by drone i, the fitness function F(x i )for:

[0108]

[0109] where a ij is an element in the adjacency matrix, x i ∈[0,1] corresponds to the strategy of drone i, x j The strategy for drone j

[0110] The difference in fitness function between drone i and drone j is:

[0111] ΔF ij =F(x i )-F(x j )

[0112] According to the imitation dynamics, each drone switches to the adjacent strategy with a certain probability, and introduces the following strategy evolution rules:

[0113]

[0114] k represents the number of iterations, deg(v i ) is the degree of the undirected graph;

[0115] pij =sig(β|ΔF ji |)

[0116]

[0117] p ij is the probability that the drone switches to an adjacent strategy. In other words, in the iteration, the drone switches to an adjacent strategy with probability p. ij Switch to the adjacent strategy, β is a constant greater than 0.

[0118] The strategy adaptation rule of drone i can be obtained as follows:

[0119]

[0120] make The above formula can be simplified as:

[0121] Step 2: Establish an incremental network generation algorithm;

[0122] Based on the strength and number of connections between nodes, networks can be classified as strongly connected, weakly connected, or disconnected. In a strongly connected network, every node can reach any other node, while in a weakly connected network, some nodes require intermediate nodes to reach other nodes, but can eventually reach all nodes. A disconnected network consists of two or more isolated subnetworks where there are no interconnected paths between nodes.

[0123] The incremental network generation algorithm converts disconnected and weakly connected networks into strongly connected networks. The process includes the following steps: First, for the current non-connected undirected graph G, use the Tarjan SCC algorithm to split it into several strongly connected components. If there is no direct path between any two of these strongly connected components, add at least one directed edge between them to make them connected to each other. Next, for each pair of strongly connected components [(S i ,S j )], if S i If it cannot be reached, then j Randomly select at least one node from S i Randomly select a node in and add directed edges between them. Repeat this step until all strongly connected components are connected to each other. The resulting network is a strongly connected graph. The pseudo code of the incremental network generation algorithm is as follows:

[0124]

[0125] Step 3: Establish a dynamic model that considers the UAV under the influence of multiple nonlinear environments;

[0126] During the interaction between drones, drones will also be affected by the environment. The dynamic model under the influence of multiple nonlinear environments can be expressed as:

[0127]

[0128] f i is the unknown nonlinear environmental impact on UAV i, f i =[sinx i ,cosx i ,…] is composed of multiple nonlinear functions, deg(v i ) is the degree of the undirected graph.

[0129] It should be noted that the behavior of drones is affected by many factors during flight. Due to the complexity of these factors, it is difficult to give an accurate mathematical description. Therefore, this paper proposes a multi-layer nonlinear CAID model. In this model, f i =[sinx i ,cosx i ,…] consists of multiple nonlinear functions. During the simulation process, each UAV randomly selects a different nonlinear function to simulate in each step.

[0130] Step 4: Introduce an incremental network with long-term guidance to deal with environmental impacts;

[0131] The impact of the environment on drones is difficult to assess. In the present invention, the participation of the leader directly affects the edge weights, thereby affecting the information exchange between drones. Specifically, if there is a leader-guided connection between two drones, then a ij >0, otherwise a ij <0.

[0132] Therefore, a long-term machine-guided incremental network with dynamic weights is introduced to cope with environmental influences. This network can be expressed as:

[0133]

[0134] η i is the effect caused by the best guidance of the leader. The neural network fitting method is used to obtain the optimal level of the leader's participation.

[0135] Traditionally, the stability of Nash equilibrium is determined by evaluating the eigenvalues ​​of the Jacobian matrix. However, this approach is highly dependent on the network structure of the system, which makes it challenging to analyze the convergence of evolutionary dynamics models of large complex networks.

[0136] Therefore, the present invention adopts Lyapunov function to solve this problem.

[0137] The dynamic model in the system is considered as:

[0138]

[0139] x(t) is the state of the system, and f(x,t) is the continuous state function of the system.

[0140] The following is the lemma:

[0141] Lemma 1: If the equilibrium is Lyapunov stable, then for all ε>0, there exists a δ>0, and if ||x0-α||<δ, then ||x(t,t0,x0)-α||≤ε,

[0142] Lemma 2: For a function: The energy function satisfies:

[0143] V(x,t)≥0,

[0144] Then the balance point: Stablize

[0145] Radial basis function neural network (RBF-NN) is a typical nonlinear modeling tool because the modeling of highly uncertain and complex systems can approximate any ongoing function. Therefore, RBF-NN can be used to fit unknown nonlinear functions as follows:

[0146]

[0147] The input vector is: X = [x1,…,x N ], weight vector: W = [W1,…,W N ], is the Gaussian function, and ζ is the fitting error of the nonlinear function.

[0148] So choose a suitable approximation: is the estimated value of the weight matrix, then the estimation error is

[0149] The estimated error of the function can be expressed as:

[0150] Based on the dynamic model of incremental network and UAV, the present invention designs the Lyapunov function to prove the convergence of the proposed model. It is mathematically proved that the proposed model is stable, which can ensure that the model can converge to an equilibrium state under any circumstances, and can also avoid the occurrence of chaos or oscillation in the model, and is not affected by parameters, making it more applicable.

[0151] In order to analyze the convergence of the CAID model in the network, the CAID model system can be described as:

[0152]

[0153] Where L k =L(G k ) is the graph G k The Laplace operator of x(t) is the state of the system.

[0154] It can be calculated by the following formula:

[0155]

[0156] in p ij is the probability of the drone switching to an adjacent strategy, a ij is an element in the adjacency matrix, deg(v i ) is the degree of the undirected graph.

[0157] Let α be the equilibrium point of the dynamic system and e be the error between state x and the equilibrium point α.

[0158] have:

[0159] x(t)=α+e

[0160] Introducing the lemma:

[0161] Lemma 3: If graph G k is strongly connected, then it has a spanning tree, and L k The rank is N-1

[0162] Lemma 4: If the rank of L is N-1, then 1α is the only vector in the null space of L that, for some constant α, has a steady-state x s =1α

[0163] According to Lemma 4: So we can get:

[0164]

[0165] And because:

[0166] e j -e i =x j (t)-x i (t)

[0167] but:

[0168] in

[0169] Ⅰ. Convergence Analysis of CAID Model in Incremental Network (Convergence Analysis of CAID Model with Incremental Network Algorithm)

[0170] In order to prove that this evolutionary dynamics model with N drones is convergent, let the Lyapunov function be:

[0171]

[0172] where e i is the drone status x i The error from the equilibrium point α;

[0173] The derivative of V is

[0174]

[0175] Depend on It can be further rewritten as:

[0176]

[0177] At this point, the proof is complete, and the CAID dynamics model with limited learning ability is convergent.

[0178] Ⅱ. Convergence Analysis of CAID Model with Multiple Unknown Nonlinear Environmental Effects

[0179] When it comes to the following kinetic models:

[0180] Will no longer be satisfied It is also impossible to calculate the error using x(t) = α + e. Therefore, in order to make the proposed incremental network with leader guidance improve the level of cooperation while the system converges, the error of UAV i is set to: e i =x i (t)-x i /

[0181] x i * is the strategy value with the maximum fitness among the neighbors of drone i, so for each time t:

[0182]

[0183] If the relationship between drones is fully connected, we can get:

[0184] x1*=x2*=…=x N * , so we have e i -e j =x i (t)-x i * -xj (t)+x j * =x i (t)-x j (t)

[0185] Assumptions:

[0186] 1) The unknown ideal NN weight matrix W is represented by ||W|| F ≤W M Boundary

[0187] 2) NN activation function Bounded, for the entire network:

[0188] Theorem 1: Consider the system given in Assumption 1, so that the communication graph is strongly connected, the optimal guidance level for selecting the leader is:

[0189] Introduction Theorem 1: Consider the system given in Assumption 1), so that the communication graph is strongly connected, the optimal guidance level for selecting the leader is:

[0190] in is a Gaussian function

[0191] At the same time, the weight matrix update law is

[0192] w ij is the element in the unknown weight matrix NN.

[0193] Then the system: convergence

[0194] where p β ij =η i p ij , η i The best level with the guidance of the captain, p ij is the probability of switching to the neighboring strategy, f i is the unknown nonlinear environmental impact on UAV i.

[0195] prove:

[0196] In order to prove that the above dynamic system is convergent in the network with the leader guidance, let the Lyapunov function be The derivative of the function is

[0197] is the estimation error of UAV i;

[0198] Make the following replacements:

[0199] where p β ij =η i p ij , η i The best level with the guidance of the captain, p ij is the probability of switching to the neighboring strategy, f i is the unknown nonlinear environmental impact on UAV i;

[0200] can be written as:

[0201] When satisfied: Can get

[0202]

[0203] therefore:

[0204]

[0205] Substitute:

[0206]

[0207] If satisfied: have:

[0208]

[0209] Corresponding:

[0210] definition:

[0211] You can get:

[0212] Based on Assumption 1, let φ β max =max(φ ij β ,φ ij β )

[0213] You can get:

[0214]

[0215] At this point, the proof is complete, and the CAID dynamics model in the incremental network with long-machine guidance is convergent.

[0216] Example 2

[0217] This example establishes a network between drones for the prisoner's dilemma and the snowdrift dilemma, and verifies the convergence of the drone's dynamic model.

[0218] The steps are as follows:

[0219] Step 1: Randomly generate a graph G with 20 drones (N=20)

[0220] Step 2: Use the incremental network generation algorithm to output a strongly connected network G'

[0221] First, for the current non-connected undirected graph G, use the strong component algorithm TarjanSCC to split it into several strongly connected components. If there is no direct path between any two of these strongly connected components, add at least one directed edge between them to make them connected to each other. Next, for each pair of strongly connected components [(S i ,S j )], if S i If it cannot be reached, then j Randomly select at least one node from S i Randomly select a node in and add directed edges between them. Repeat this step until all strongly connected components are connected to each other.

[0222] The pseudo code is as follows:

[0223]

[0224] Step 3: Build a UAV dynamics model and set CAID model parameters for two specific examples: the prisoner’s dilemma and the snowdrift dilemma

[0225] Prisoner’s Dilemma:

[0226] CAIPD: CAIPD corresponds to a classic example in game theory: the prisoner's dilemma. It represents a non-zero-sum game. This game reflects that what may be the best choice for an individual may not be the best choice for the group. In the prisoner's dilemma game, the best individual strategy is to defect. However, if everyone chooses to defect, then everyone will be worse off, resulting in a conflict between individual interests and group interests.

[0227] The payoff matrix is:

[0228]

[0229] Among them, b is the benefit obtained by the individual, c is the contribution of the collaborator, and satisfying b <c。

[0230] The fitness function of drone i is:

[0231] The difference in fitness function between drone i and drone j is:

[0232]

[0233] Where: p ij =sig(β|ΔF ji |), deg(v i ) is the degree of the undirected graph.

[0234] Snowdrift Dilemma:

[0235] CAISD: CAISD corresponds to the snowdrift dilemma: two drivers are blocked by snow. Let the cost of clearing the snowdrift to make the road clear be c. If the road is clear, the benefit to each person is quantified as b. If both drivers work together to shovel snow, their benefits are R = bc / 2; if only one person shovels snow, although both can go home, the betrayer escapes labor, and his benefits are T = b, while the benefits of the cooperator are S = bc; if both drivers choose not to cooperate, both drivers are blocked by the snowdrift and cannot go home, and their benefits are both P = 0.

[0236] The profit matrix can be expressed as:

[0237]

[0238] The fitness function of drone i is:

[0239]

[0240] The difference in fitness function between drone i and drone j is:

[0241]

[0242] Where: p ij =sig(β|ΔF ji |), deg(v i ) is the degree of the undirected graph.

[0243] The parameters of CAID are set to b=5, c=1, β=1

[0244] Step 4: Verify the convergence of the CAID model on graph G and graph G' respectively:

[0245] like Figure 5 , the CAIPD model is Figure 3 The unconnected graph in does not converge. When the incremental network algorithm is used, the following is obtained: Figure 4 The strongly connected network in the model also converges ( Figure 6 );

[0246] like Figure 7 , the CAISD model Figure 3 The unconnected graph in does not converge. When the incremental network algorithm is used, the following is obtained: Figure 4 The strongly connected network in the model also converges ( Figure 8 );

[0247] Step 5: Verify the role of the incremental network with long-term guidance under multiple nonlinear environmental influences:

[0248] Select f i is the unknown nonlinear environmental impact on UAV i, f i =[sinx i ,cosx i ,…] consists of multiple nonlinear functions, and introduces an incremental network with dynamic weights and leader guidance to deal with environmental influences. If there is leader guidance connectivity between two UAVs, a ij <0, otherwise a ij = 0. Therefore, an incremental network with long-term guidance is introduced to counter the influence of the environment.

[0249] It can be expressed as:

[0250]

[0251] η i In order to determine the impact of the optimal guidance of the lead aircraft, a neural network fitting method is used to obtain the optimal level of the lead aircraft's guidance.

[0252] like Fig.10 , the incremental network with long-term guidance can avoid the negative impact caused by environmental impact in the CAIPD model, where environmental impact will make Fig. 9 It is difficult for the model to achieve global convergence. Fig.11 and Fig.12 In comparison, the incremental network with long-term machine guidance can also accelerate the global convergence of the CAISD model, which further demonstrates the effectiveness of the incremental network with long-term machine guidance.

[0253] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A UAV learning method with the guidance of a leader based on an incremental network, characterized in that: The following steps are involved: Step 1: Establish an iterative model of drones based on different environments and establish a profit matrix; Step 2: Establish an incremental network generation algorithm. For a non-connected undirected graph G, use the Tarjan SCC algorithm to split it into several strongly connected components; for each pair of strongly connected components [(S i ,S j )], if S i If it cannot be reached, then j Randomly select at least one node from S i Randomly select a node in and add directed edges between them, repeating this step until a strongly connected graph is generated; Step 3: Establish a dynamic model that considers the UAV under the influence of multiple nonlinear environments; Step 4: Introduce an incremental network with long-term guidance to deal with environmental impacts.

2. The method for learning unmanned aerial vehicles with the guidance of a leader based on an incremental network according to claim 1, characterized in that: The step 1 further comprises the following steps: Step 1.1 Establish a UAV dynamics model based on CAID; Set x i ∈[0,1] corresponds to the strategy of drone i, where i∈{1,2,…,N} represents a drone, where x=0 represents betrayal and x=1 represents full cooperation; Step 1.2 Establish the model profit matrix; The payoff matrix is: Where μ0, μ1, μ2, and μ3 represent the benefits of each participant in the traditional evolutionary dynamics model; Step 1.3 Establish the model fitness function For the strategy selected by drone i, the fitness function F(x i )for: where a ij is an element in the adjacency matrix of the undirected graph G, x i ∈[0,1] corresponds to the strategy of drone i, x j The corresponding strategy of drone j.

3. The method for learning unmanned aerial vehicles with the guidance of a leader based on an incremental network according to claim 1, characterized in that: In step 3, the dynamic model under the influence of multiple nonlinear environments can be expressed as: p ij is the probability of switching to the neighboring strategy, f i is the unknown nonlinear environmental impact on UAV i, f i =[sinx i ,cos x i ,…] consists of multiple nonlinear functions, a ij is an element in the adjacency matrix of the undirected graph G, deg(v i ) is the degree of the undirected graph.

4. The method for learning unmanned aerial vehicles with the guidance of a leader based on an incremental network according to claim 1, characterized in that: In step 4, a dynamic weighted incremental network with a long-machine guidance is introduced to cope with environmental impacts. The dynamic model can be expressed as: η i is the effect of the best guidance from the leader, a ij is an element in the adjacency matrix of the undirected graph G, f i is the unknown nonlinear environmental impact on UAV i, f i =[sin x i ,cos x i ,…] consists of multiple nonlinear functions, p ij is the probability of switching to the neighboring strategy.

5. The method for learning unmanned aerial vehicles with the guidance of a leader based on an incremental network according to claim 4 is characterized in that: The radial basis function neural network is used to fit the dynamic model of the incremental network with long-term guidance to cope with environmental influences, as follows: The input vector is: X = [x1,…,x N ], weight vector: W = [W1,…,W N ], is the Gaussian function, ζ is the fitting error of the nonlinear function; So choose a suitable approximation: is the estimated value of the weight matrix, then the estimation error is The estimated error of the function can be expressed as:

6. The method for learning an unmanned aerial vehicle with the guidance of a leader based on an incremental network according to any one of claims 1 to 5, characterized in that: It also includes model convergence analysis, during which the lemma is introduced: Lemma 3: If graph G k is strongly connected, then it has a spanning tree, and L k The rank is N-1 Lemma 4: If the rank of L is N-1, then 1α is the only vector in the null space of L that, for some constant α, has a steady-state x s =1α.

7. The method for learning unmanned aerial vehicles with the guidance of a leader based on an incremental network according to claim 6, characterized in that: The convergence analysis method of the CAID model in the incremental network obtained in step 3 is as follows: Assume the Lyapunov function is: where e i is the drone status x i The error from the equilibrium point α; The derivative of V is Depend on It can be further rewritten as: The proof is that the CAID dynamics model with limited learning ability is convergent, where in 8. The method for learning unmanned aerial vehicles with the guidance of a leader based on an incremental network according to claim 5, characterized in that: The convergence analysis method of the CAID model with multiple unknown nonlinear environmental influences obtained in step 4 is as follows: Assumptions: 1) The unknown ideal NN weight matrix W is represented by ||W|| F ≤W M Boundary 2) NN activation function Bounded, for the entire network: Introduction Theorem 1: Consider the system given in Assumption 1), so that the communication graph is strongly connected, the optimal guidance level for selecting the leader is: in is a Gaussian function At the same time, the weight matrix update law is w ij is the element in the unknown weight matrix NN; Then the system: convergence where p β ij =η i p ij , η i The best level with the guidance of the captain, p ij is the probability of switching to the neighboring strategy, f i is the unknown nonlinear environmental impact on UAV i; prove: Let the Lyapunov function be The derivative of the function is is the estimation error of UAV i; Make the following replacements: where p β ij =η i p ij , η i The best level with the guidance of the captain, p ij is the probability of switching to the neighboring strategy, f i is the unknown nonlinear environmental impact on UAV i; can be written as: When satisfied: Can get therefore: Substitute: If satisfied: have: Corresponding: definition: You can get: Based on Assumption 1, let φ β max =max(φ ij β ,φ ij β ) You can get: At this point, the proof is complete, and the CAID dynamics model in the incremental network with long-machine guidance is convergent.

Images