Multi-unmanned aerial vehicle formation distributed optimization fault-tolerant control method and system

Abstract

The invention relates to the technical field of unmanned aerial vehicle formation control, and provides a multi-unmanned aerial vehicle formation distributed optimization fault-tolerant control method and system, and the method comprises the following steps: building a nonlinear dynamic model in advance when an unmanned aerial vehicle formation breaks down; a distributed unmanned aerial vehicle communication topological structure is established in advance; when the unmanned aerial vehicle formation has a fault, determining current unmanned aerial vehicle control input, the position and speed of the unmanned aerial vehicle in the current formation, and trajectory information of the pilot unmanned aerial vehicle in the formation; determining a global tracking error of the unmanned aerial vehicle formation according to the position information between the adjacent unmanned aerial vehicles; calculating a sliding mode surface; calculating sliding mode position control input; calculating optimization control input; and performing control input adjustment of the unmanned aerial vehicle according to the position control input and the optimization control input. According to the technical scheme of the invention, even if one or more unmanned aerial vehicles in the formation system break down, the unmanned aerial vehicle formation can still complete the communication of task information based on the information transmission of the adjacent unmanned aerial vehicles and keep a stable formation structure.

Description

Technical Field

[0001] The present invention relates to the technical field of multi - UAV formation control, and more specifically, to a distributed optimization fault - tolerant control method and system for multi - UAV formation. Background Art

[0002] With the increasing demand for efficient execution of complex tasks by UAVs, multi - UAV collaborative operations have been widely used in various operation scenarios due to their flexibility and efficiency. However, with the improvement of task complexity, UAV accidents occur frequently. Especially with the further popularization of UAVs in the civilian field, the safety issues of UAVs have become increasingly prominent.

[0003] When one or more UAVs in a UAV formation are affected by uncertain factors such as environmental interference, communication loss, and actuator failures, it is easy to cause the overall stability of the formation to decline, resulting in problems such as low task - completion efficiency. Moreover, the control methods currently adopted by formation systems mainly adjust the position and attitude based on information exchange between adjacent UAVs. When a UAV fails, the fault information will be transmitted to adjacent UAVs, which is extremely likely to cause a chain reaction, threatening the overall stability of the group and even leading to the collapse of the entire formation system. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art or related technologies.

[0005] To this end, the object of the present invention is to provide a distributed optimization fault - tolerant control method and system for multi - UAV formation. Considering actuator failures, modeling uncertainties, and external disturbance factors for each UAV in the formation system, a non - linear dynamic model of the UAV is constructed. A communication strategy based on the Leader - Follower architecture and distributed control method is adopted to ensure the efficient transmission of task information between UAVs. A fault - tolerant controller based on integral sliding mode and neural network is designed. At the same time, a reinforcement learning strategy based on an execution network and an evaluation network is designed to optimize the control input, ensuring that even if one or more UAVs in the formation system fail under the guidance of the leader UAV, the UAV formation can still complete the communication of task information based on the information transmission between adjacent UAVs and maintain a stable formation structure.

[0006] To achieve the above object, the first - aspect technical solution of the present invention provides a distributed optimization fault - tolerant control method for multi - UAV formation, including the following steps:

[0007] S1, pre - construct a non - linear dynamic model when the UAV formation fails. The non - linear dynamic model includes:

[0008] where \(i = 1,2,\cdots,N\) represents the sequential number of the UAVs in the formation system, \(p\) i (t)=[x i (t),y i (t)] T , \(v\) i (t)=[v xi (t),v yi (t)] T , [·] T represents the transpose of a matrix, and respectively represent taking the first-order derivatives of the UAV position and velocity, \(u\) i (t)=u si (t)+u oi (t), \(u\) si (t) represents the sliding-mode position control input signal of the UAV in the formation system, \(u\) oi (t) represents the optimal control input, \(u\) i represents the UAV control input in the current formation, \(f\) i1 (p i ,v i ) represents the system uncertainty, represents the UAV speed, \(p\) i (t) represents the UAV position, \(D\) i (t) represents the external disturbance acting on the UAV, \(\rho\) represents the efficiency factor of the actuator, and \(f\) i1 (p i ,v i ) and \(D\) i (t) values are preset;

[0009] S2. According to the formation mission requirements, a distributed UAV communication topology is established in advance, and a leader-follower information transfer method is adopted;

[0010] S3. When a UAV formation fails, according to the non-linear dynamics model, determine the current UAV control input \(u\) fi , the position and velocity \(p\) i =[x i ,y i T ,v i =[v xi ,v yi T , the trajectory information \(p\) 0i =[x 0i ,y 0i T ,v 0i =[v x0i ,v​​​y0i T ;

[0011] S4. Determine the global position tracking error \(z_i(t)\) and the global velocity tracking error \(\dot{z}_i(t)\) of the \(i\)-th unmanned aerial vehicle (UAV) in the formation system under the interaction of formation UAVs respectively; ip (t) and the global velocity tracking error \(z\) iv (t);

[0012] S5. Determine the global tracking error \(e(t)\) of the UAV formation according to the position information between adjacent UAVs; i (t);

[0013] S6. Calculate the sliding mode surface \(s(t)\) according to the global tracking error, the position tracking error, and the velocity tracking error according to the pre-designed integral sliding mode; i (t);

[0014] S7. Calculate the sliding mode position control input \(u(t)\) according to the sliding mode surface, the neural network weight adaptation law, and the position control input model; si (t);

[0015] S8. Calculate the optimal control input \(u^*(t)\) according to the execution network weight adaptation law, the evaluation network weight adaptation law, the global tracking error, and the optimal control input model; oi (t);

[0016] S9. Adjust the control input of the UAV according to the position control input and the optimal control input according to the nonlinear dynamics model.

[0017] In the above technical solution, preferably, step S4 specifically includes the following steps:

[0018] Pre-establish a distributed position and velocity tracking error model of the global system based on the interaction of adjacent UAVs according to the UAV communication topology;

[0019] Calculate and determine the global position tracking error and the global velocity tracking error of the \(i\)-th UAV in the formation system under the interaction of formation UAVs respectively according to the distributed position and velocity tracking error model,

[0020] wherein, the distributed position and velocity tracking error model includes:

[0021]

[0022] where \(i = 1, 2, \cdots, n\), representing the serial number of the UAV in the formation system, \(j\) represents the UAV adjacent to the \(i\)-th UAV, \(z_i(t)\) and \(\dot{z}_i(t)\) respectively represent the global position tracking error and the global velocity tracking error of the \(i\)-th UAV considering the information interaction between UAVs in the formation, ip (t) and \(z\) iv (t) respectively represent the global position tracking error and the global velocity tracking error of the \(i\)-th UAV considering the information interaction between UAVs in the formation,​ Characterize the position tracking error of the i-th UAV Characterize the position tracking error of the j-th UAV adjacent to the i-th UAV in the formation system, p i Characterize the position of the i-th UAV, p 0 Characterize the trajectory information of the leader UAV, h ip Characterize the structure of the formation shape to be maintained Characterize the speed tracking error of the i-th UAV, v i Characterize the speed of the i-th UAV, v 0 Characterize the trajectory information of the leader UAV, h iv Characterize the structure of the formation shape to be maintained Characterize the speed tracking error of the j-th UAV adjacent to the i-th UAV in the formation system, where i≠j, and there is also z ip The derivative of is equal to z iv Such a relationship, that is a ij Characterize the element in the adjacency matrix A, b i Characterize the element in the degree matrix D, adjacency matrix Degree matrix

[0023] In any of the above technical solutions, preferably, in step S5, the global tracking error calculation model of the UAV formation is:

[0024]

[0025] Where, e i (t) Characterize the global tracking error Characterize the position tracking error of the i-th UAV Characterize the position tracking error of the j-th UAV adjacent to the i-th UAV in the formation system, p i Indicate the position of the i-th UAV, p 0 Characterize the trajectory information of the leader UAV Characterize the speed tracking error of the i-th UAV, v i Characterize the speed of the i-th UAV, v 0 Characterize the trajectory information of the leader UAV, h iv Characterize the structure of the formation shape to be maintained

[0026] In any of the above technical solutions, preferably, in step S6, the pre-designed integral sliding mode is:

[0027]

[0028] where s i (t) is characterized as the sliding mode surface, e i (t) is characterized as the global tracking error, e i (0) is characterized as the initial state error, Ξ i = [z iv (t), -v 0 -h iv T is characterized as a matrix containing z iv (t) and -v 0 -h iv where z iv (t) is characterized as the global velocity tracking error, G i = [0 n*n , I n is characterized as a constant matrix, I n is characterized as the n-dimensional identity matrix, is an m-dimensional matrix satisfying KG i is invertible.

[0029] In any of the above technical solutions, preferably, in step S7, the position control input model is:

[0030] where μ i , k si are characterized as positive constant gains, sign is the sign function, φ di (x i ) is characterized as the selected basis function vector, is the estimated value of the neural network weight, (·) -1 is the inverse matrix of the matrix, s i (t) is the sliding mode surface;

[0031] The neural network weight adaptation law is:

[0032] where k d1 , k d2 are characterized as positive constant gains, is the transpose of the sliding mode surface s i (t), φ di (x i ) is the basis function vector related to the system state.

[0033] In any of the above technical solutions, preferably, step S8 specifically includes the following steps:

[0034] Convert the global system tracking error e i (t) into a nominal error system, ​

[0035] Among them, U oi = G i u oi represents the control input of the i-th unmanned aerial vehicle (UAV) under the nominal error system,

[0036] According to the optimized nominal error system, a performance index function is established.

[0037] Among them, J(0) represents the performance index. represents the cost function in the formation system, e i (t) represents the global system tracking error, U oi (t) represents the control input of the i-th UAV under the nominal error system. represents the transpose of the control input of the i-th UAV under the nominal error system, represents the transpose of the global system tracking error.

[0038] According to the performance index function, the optimal control input under the nominal error system is determined. Specifically,

[0039]

[0040] Among them, represents the performance index under the nominal system, represents the optimal control input, min∫(·) represents the minimum value of the performance index,

[0041] An optimized control input model is constructed, specifically:

[0042] Among them, μ oi represents a positive constant to be designed, U oi = G i u oi (t), G i = [0 n*n , I n , I n represents the n-dimensional identity matrix, represents the estimated value of the execution network weight, φ oi (e i ) represents the basis function vector related to the global tracking error of the UAV formation;

[0043] The estimated value of the execution network weight is determined according to the execution network weight adaptation law, and the execution network weight adaptation law is:

[0044]

[0045] Among them, kai , k ci is characterized as a normal constant gain is characterized as the estimated value of the evaluation network weights is characterized as the first-order time derivative of φ oi (e i ) is characterized as the basis function vector related to the global tracking error of the UAV formation is characterized as the transpose of the basis function vector;

[0046] The estimated value of the evaluation network weights is determined according to the evaluation network weight adaptation law, and the evaluation network weight adaptation law is:

[0047]

[0048] where k ci is characterized as a normal constant gain, φ oi (e i ) is characterized as the basis function vector related to the global tracking error of the UAV formation is characterized as the transpose of the basis function vector, is characterized as the first-order time derivative of

[0049] According to the optimized control input model, iterative convergence is performed to calculate the optimized control input u oi (t).

[0050] The technical solution of the second aspect of the present invention proposes a multi-UAV formation distributed optimized fault-tolerant control system, which is communicatively connected to the UAV formation and realizes the steps of any one of the multi-UAV formation distributed optimized fault-tolerant control methods proposed in the technical solution of the first aspect of the present invention as described above.

[0051] A multi-UAV formation distributed optimized fault-tolerant control method and system proposed by the present invention have the following beneficial technical effects:

[0052] (1) A multi-UAV formation distributed optimized fault-tolerant control method and system proposed by the present invention can significantly improve the response ability of the multi-UAV formation system when a single UAV fails, effectively reduce the collapse of the formation system or the loss of mission information caused by UAV failures, and have high reliability and robustness, which is beneficial to promoting the rapid development of the UAV industry.

[0053] (2) A distributed optimal fault-tolerant control method and system for multi-UAV formation proposed by the present invention locates the external disturbances and actuator faults existing in the formation system as the lumped uncertainty of the system, and uses the sliding mode control method and the neural network method to design the integral sliding mode and the neural network adaptive law to achieve effective approximation, which can ensure that after single-point or multi-point faults occur in the formation system, the overall stability of the formation will not be affected, and the effective tracking of the target and the consensus of the formation global system can be realized.

[0054] (3) A distributed optimal fault-tolerant control method and system for multi-UAV formation proposed by the present invention transforms the global system tracking error into a nominal error system based on the integral sliding mode, and establishes a performance index function based on this nominal error system, transforming the tracking problem of the UAV formation system into an optimal control problem, that is, solving the Hamilton-Jacobi-Bellman equation, and using the reinforcement learning control method based on the execution network and the evaluation network, so that the system parameters can be fully trained to obtain the optimal control input, making full use of the computing resources and optimizing the overall formation control efficiency.

[0055] (4) A distributed optimal fault-tolerant control method and system for multi-UAV formation proposed by the present invention effectively combines the optimal control method and the sliding mode control method, can effectively handle the influences brought by actuator faults, modeling uncertainties, external disturbance factors, etc., and the adaptive law is simpler and easier to deploy, improving the resource utilization efficiency of large-scale formation flight and realizing the effective execution of optimal formation control.

[0056] (5) A distributed optimal fault-tolerant control method and system for multi-UAV formation proposed by the present invention considers actuator faults, modeling uncertainties, and external disturbance factors for each UAV in the formation system, constructs the nonlinear dynamic model of the UAV, adopts the communication strategy based on the Leader-Follower architecture and the distributed control method to ensure the efficient transmission of task information between UAVs, designs the fault-tolerant controller based on the integral sliding mode and the neural network, and at the same time, designs the reinforcement learning strategy based on the execution network and the evaluation network to optimize the control input, ensuring that even if single or multiple UAVs in the formation system fail under the guidance of the leader UAV, the UAV formation can still complete the communication of task information based on the information transmission of adjacent UAVs and maintain a stable formation structure.

[0057] The additional aspects and advantages of the present invention will be given in the following description part, some will become obvious from the following description, or will be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The above and / or additional aspects and advantages of the present invention will become obvious and easy to understand from the description of the embodiments in conjunction with the following drawings, where:

[0059] Figure 1 Shows a schematic flow chart of a fault-tolerant method for UAV formation according to an embodiment of the present invention;

[0060] Figure 2 Shows a schematic diagram of a distributed UAV communication topology according to an embodiment of the present invention;

[0061] Figure 3 Shows an algorithm architecture diagram of a fault-tolerant method for UAV formation according to an embodiment of the present invention;

[0062] Figure 4 Shows a three-dimensional flight trajectory diagram of multiple UAVs with a fault-tolerant algorithm according to an embodiment of the present invention;

[0063] Figure 5 Shows the execution network Ψ ai The norm of the weight ||Ψ ai ||;

[0064] Figure 6 Shows the evaluation network Ψ ci The norm of the weight ||Ψ ci ||;

[0065] Figure 7 Shows the neural network Ψ di The norm of the weight ||Ψ di ||;

[0066] Figure 8 Shows the speed tracking error of the UAV. Detailed implementation manners

[0067] In order to more clearly understand the above objects, features and advantages of the present invention, the present invention will be further described in detail below with reference to the drawings and specific implementation manners. It should be noted that, without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other.

[0068] In the following description, many specific details are set forth in order to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0069] The following combines Figures 1 to 8 Specifically describes a fault-tolerant control method and system for UAV formation according to an embodiment of the present invention.

[0070] As Figure 1 shown, a fault-tolerant control method for UAV formation according to an embodiment of the present invention includes the following steps:

[0071] S101. Pre - construct a non - linear dynamic model for the UAV formation in case of failure. The non - linear dynamic model includes:

[0072] where \(i = 1,2,\cdots,N\) represents the sequential number of UAVs in the formation system, \(p\) i (t)=[x i (t),y i (t)] T , \(v\) i (t)=[v xi (t),v yi (t)] T , [·] T represents the transpose of a matrix, and respectively represent taking the first - order derivative of the UAV position and velocity. \(u\) i (t)=u si (t)+u oi (t), \(u\) si (t) represents the sliding - mode position control input signal of the UAV in the formation system, \(u\) oi (t) represents the optimal control input, \(u\) i represents the UAV control input in the current formation, \(f\) i1 (p i ,v i ) represents the system uncertainty, represents the UAV velocity, \(p_i(t)\) represents the UAV position, \(D\) i (t) represents the external disturbance suffered by the UAV, \(\rho\) represents the efficiency factor of the actuator. Preset the values of \(f\) i1 (p i ,v i ) and \(D\) i (t);

[0073] S102. According to the formation mission requirements, pre - establish a distributed UAV communication topology structure and adopt a leader - follower information transfer method;

[0074] S103. When the UAV formation fails, according to the non - linear dynamic model, determine the current UAV control input \(u\) fi , the position and velocity \(p\) i =[x i ,y i T , \(v\) i =[v xi ,v yi T , the trajectory information \(p\) 0i =[x​​0i , y 0i T , v 0i = [v x0i , v y0i T ;

[0075] S104. Determine the global position tracking error z ip (t) and the global velocity tracking error z iv (t) of the i-th unmanned aerial vehicle in the formation system under the interaction of formation unmanned aerial vehicles respectively;

[0076] S105. Determine the global tracking error e i (t) of the unmanned aerial vehicle formation according to the position information between adjacent unmanned aerial vehicles;

[0077] S106. Calculate the sliding mode surface s i (t) according to the global tracking error, position tracking error and velocity tracking error according to the pre-designed integral sliding mode;

[0078] S107. Calculate the sliding mode position control input u si (t) according to the sliding mode surface, neural network weight adaptation law and position control input model;

[0079] S108. Calculate the optimal control input u oi (t) according to the execution network weight adaptation law, evaluation network weight adaptation law, global tracking error and optimal control input model;

[0080] S109. Adjust the control input of the unmanned aerial vehicle according to the position control input and the optimal control input according to the nonlinear dynamics model.

[0081] ​​Considering actuator faults, modeling uncertainties, and external disturbance factors for each UAV in the formation system, a nonlinear dynamics model of the UAV is constructed. A communication strategy based on the Leader-Follower architecture and distributed control method is adopted to ensure the efficient transmission of task information among UAVs. Based on the distributed communication protocol between adjacent UAVs, the tracking error of the formation global system is calculated. Through the design of integral sliding mode and neural network, the faults of single UAVs in the system and the lumped uncertainties of the system are solved. The control input is optimized through the reinforcement learning strategy of the execution network and the evaluation network, making full use of computing resources to achieve the optimization of formation control. Thus, it is ensured that under the guidance of the leader UAV, even if one or more UAVs in the formation system fail, the UAV formation can still complete the communication of task information based on the information transmission between adjacent UAVs and maintain a stable formation structure. Furthermore, the response ability of the multi-UAV formation system when a single UAV fails is significantly improved, and the collapse of the formation system or the loss of task information caused by UAV faults can be effectively reduced.

[0082] Furthermore, determine the UAV communication topology structure as Figure 2 shown, adopt the leader-follower information transmission method, and establish a distributed position and velocity tracking error model of the global system based on the interaction between adjacent UAVs.

[0083] The distributed position and velocity tracking error model includes:

[0084]

[0085] where i = 1, 2,..., n, representing the sequential number of UAVs in the formation system, j represents the UAV adjacent to the ith UAV, z ip (t) and z iv (t) respectively represent the global position tracking error and global velocity tracking error of the ith UAV considering the information interaction between UAVs in the formation, represents the position tracking error of the ith UAV, represents the position tracking error of the jth UAV adjacent to the ith UAV in the formation system, p i represents the position of the ith UAV, p 0 represents the trajectory information of the leader UAV, h ip represents the structure of the formation shape to be maintained, represents the velocity tracking error of the ith UAV, v i represents the velocity of the ith UAV, v 0 represents the trajectory information of the leader UAV, h iv represents the structure of the formation shape to be maintained, Denote the velocity tracking error of the j-th UAV adjacent to the i-th UAV in the formation system, where i ≠ j, and there is also z ip whose derivative is equal to z iv such a relationship that a ij is denoted as an element in the adjacency matrix A, and b i is denoted as an element in the degree matrix D. The adjacency matrix The degree matrix

[0086] According to the distributed position and velocity tracking error model, calculate and determine the global position tracking error and global velocity tracking error of the i-th UAV in the formation system under the interaction of formation UAVs respectively.

[0087] Thus, it further ensures that under the guidance of the leader UAV, even if one or more UAVs in the formation system fail, the UAV formation can still complete the communication of task information based on the information transmission of adjacent UAVs and maintain a stable formation structure.

[0088] Furthermore, based on the position information between adjacent UAVs, determine the global tracking error of the formation system. The calculation model of the global tracking error of the UAV formation is:

[0089]

[0090] where e i (t) is denoted as the global tracking error, denotes the position tracking error of the i-th UAV, denotes the position tracking error of the j-th UAV adjacent to the i-th UAV in the formation system, p i represents the position of the i-th UAV, p 0 is denoted as the trajectory information of the leader UAV, denotes the velocity tracking error of the i-th UAV, v i represents the velocity of the i-th UAV, v 0 is denoted as the trajectory information of the leader UAV, h iv is denoted as the structure of the formation shape to be maintained.

[0091] Thus, it is beneficial to perform corresponding control adjustments according to the global tracking error of the formation system, further ensuring the accuracy and stability of the control.

[0092] Furthermore, design an integral sliding mode according to the global tracking error of the UAV formation,

[0093]

[0094] where s i(t) is characterized as the sliding mode surface, e i (t) is characterized as the global tracking error, e i (0) is characterized as the initial state error, Ξ i = [z iv (t), -v 0 -h iv T is characterized as a matrix containing z iv (t) and -v 0 -h iv where z iv (t) is characterized as the global velocity tracking error, G i = [0 n*n , I n is characterized as a constant matrix, I n is characterized as the n-dimensional identity matrix, is an m-dimensional matrix satisfying KG i is invertible.

[0095] According to the above integral sliding mode, design the control input model,

[0096]

[0097] where μ i , k si are characterized as positive constant gains, sign is characterized as the sign function, φ di (x i ) is characterized as the selected basis function vector, is characterized as the estimated value of the neural network weight, (·) -1 is characterized as the inverse matrix of the matrix, s i (t) is characterized as the sliding mode surface;

[0098] The neural network weight adaptation law is:

[0099] where k d1 , k d2 are characterized as positive constant gains, is characterized as the transpose of the sliding mode surface s i (t), φ di (x i ) is characterized as the basis function vector related to the system state.

[0100] ​Therefore, the external disturbances and actuator faults existing in the formation system are localized as the lumped uncertainty of the system. The integral sliding mode and neural network adaptive laws are designed by using the sliding mode control method and neural network method to achieve effective approximation, which can ensure that the overall stability of the formation is not affected after single-point or multi-point faults occur in the formation system, and realize the effective tracking of the target and the consensus of the formation global system. The design of this fault-tolerant controller can not only effectively suppress the faults, but also offset the uncertainties existing in the formation system, further improve the formation control efficiency, and maintain the stability of the formation.

[0101] Furthermore, the global system tracking error e i (t) is transformed into a nominal error system,

[0102]

[0103] where U oi =G i u oi represents the control input of the i-th UAV under the nominal error system,

[0104] According to the optimized nominal error system, a performance index function is established,

[0105]

[0106] where J(0) represents the performance index, represents the cost function in the formation system, e i (t) represents the global system tracking error, U oi (t) represents the control input of the i-th UAV under the nominal error system, represents the transpose of the control input of the i-th UAV under the nominal error system, represents the transpose of the global system tracking error,

[0107] According to the performance index function, the optimal control input under the nominal error system is determined Specifically,

[0108]

[0109] where, represents the performance index under the nominal system, represents the optimal control input, min∫(·) represents the minimum value of the performance index,

[0110] An optimized control input model is constructed, specifically:

[0111] where μ oiDenoted as a positive constant \(G\) to be designed i =[0 n*n ,I n ,I n Denoted as an \(n\)-dimensional identity matrix Denoted as the estimated value of the execution network weights \(\varphi\) oi (e i ) Denoted as the basis function vector related to the global tracking error of the UAV formation

[0112] The estimated value of the execution network weights is determined according to the execution network weight adaptation law, and the execution network weight adaptation law is

[0113]

[0114] where \(k\) ai ,k ci Denoted as positive constant gains Denoted as the estimated value of the evaluation network weights Denoted as The first-order time derivative of \(\varphi\) oi (e i ) Denoted as the basis function vector related to the global tracking error of the UAV formation Denoted as the transpose of the basis function vector

[0115] The estimated value of the evaluation network weights is determined according to the evaluation network weight adaptation law, and the evaluation network weight adaptation law is

[0116]

[0117] where \(k\) ci Denoted as positive constant gains, \(\varphi\) oi (e i ) Denoted as the basis function vector related to the global tracking error of the UAV formation Denoted as the transpose of the basis function vector Denoted as The first-order time derivative of

[0118] According to the optimal control input model, iterate and converge to calculate the optimal control input \(u\) oi (t).

[0119] Therefore, it is possible to make full use of computing resources, optimize the overall formation control efficiency, and solve the optimal control of multi-UAV systems based on the framework of the execution network and evaluation network of reinforcement learning. This framework effectively approximates the negative gradient of the Hamilton-Jacobi-Bellman equation and fully trains the adaptive parameters without the condition of persistent excitation. The global tracking error of the formation system is converted into the nominal error of the formation system, that is, after the disturbances and the lumped uncertainties of the system are effectively compensated, the distributed tracking error of the formation system. Design the adaptive law of the execution network-evaluation network, which can be fully trained to make the Hamilton-Jacobi-Bellman equation converge to 0. Furthermore, it ensures that the position tracking error and speed tracking error of the global formation system are semi-globally uniformly bounded.

[0120] As Figure 3 shown, an algorithm architecture diagram of a UAV formation fault tolerance method according to an embodiment of the present invention includes S301, obtaining a distributed tracking error according to the UAV formation situation; S302, obtaining a nominal error from the distributed tracking error, S303, obtaining a performance index from the nominal error; S304, obtaining a sliding mode control variable from the distributed tracking error; S305, sliding mode dynamics; S306, reinforcement learning, based on the sliding mode dynamics, the UAV formation situation, and the performance index, reinforcement learning based on a neural network, an evaluation network, and an execution network; S307, optimal control, generated by the reinforcement learning and used to control the UAV formation.

[0121] Considering actuator faults, modeling uncertainties, and external disturbance factors for each UAV in the formation system, a nonlinear dynamic model of the UAV is constructed. A communication strategy based on the Leader-Follower architecture and distributed control method is adopted to ensure the efficient transmission of task information among UAVs. A fault-tolerant controller based on integral sliding mode and neural network is designed. At the same time, a reinforcement learning strategy based on an execution network and an evaluation network is designed to optimize the control input, ensuring that even if one or more UAVs in the formation system fail under the guidance of the leader UAV, the UAV formation can still complete the communication of task information based on the information transmission of adjacent UAVs and maintain a stable formation structure.

[0122] For the simulation experiment of the multi-UAV formation distributed optimization fault tolerance control method proposed by the present invention, the verification steps are specifically as follows. The analysis method based on Lyapunov stability can prove that when time approaches infinity, the distributed position tracking error and speed tracking error asymptotically converge to 0 respectively.

[0123] The relevant parameter values involved are as follows:

[0124] Set the trajectory of the leader UAV in the formation as p 0 =[0.1t, 0.1t] T, the formation vector of the i-th unmanned aerial vehicle is defined as h ip =[r cos(ωt + π(i - 1) / 2), r sin(ωt + π(i - 1) / 2)] T , where the radius r = 2m and the angular velocity ω = 0.5m / s.

[0125] The system uncertainty is defined as: f i (p i , v i ) = 0.15 sin(p i ) + 0.1 sin(v i ), and the external disturbance is defined as: D i =[0.01 sin(0.06πt), 0.01 sin(0.02πt)] T .

[0126] μ si = 0.85, μ oi = 42, k d1 = 0.08, k d2 = 12, k s1 = 0.02, and the initial weight values of the execution network and the evaluation network are set to The gain parameters are set to k a = 10, k c = 6. The simulation results are shown in Figure 2 、 Figures 4 to 8 respectively.

[0127] As shown in Figure 2 , a communication topology graph of the multi-unmanned aerial vehicle formation is established through the leader-follower and distributed control methods. The graph includes four follower unmanned aerial vehicles and one leader unmanned aerial vehicle, and the leader unmanned aerial vehicle can be set as a virtual leader unmanned aerial vehicle; the first unmanned aerial vehicle and the fourth unmanned aerial vehicle can receive task information from the leader unmanned aerial vehicle, and at the same time, the four unmanned aerial vehicles can exchange information with adjacent unmanned aerial vehicles.

[0128] As shown in Figure 4 , the formation unmanned aerial vehicles can fly according to the predetermined trajectory and tasks, and at the same time, the unknown states of the four unmanned aerial vehicles at the 20th second and the end of the simulation are intercepted. It can be clearly seen that the four unmanned aerial vehicles can maintain a quadrilateral formation.

[0129] As shown in Figure 5 and Figure 6 , the weights of the execution network and the evaluation network can be guaranteed to be bounded.

[0130] As shown in Figure 7 and Figure 8As shown, the distributed tracking error in the formation system can achieve effective convergence, and after injecting a fault into the third unmanned aerial vehicle (UAV) in the formation at the 15th second, the states of the four UAVs can achieve rapid stability.

[0131] It can be seen that the multi-UAV formation distributed optimization fault-tolerant control method proposed by the present invention has excellent reliability and robustness.

[0132] The steps in the method of the present invention can be adjusted, combined, and deleted in sequence according to actual needs.

[0133] The units in the device of the present invention can be combined, divided, and deleted according to actual needs.

[0134] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and this program can be stored in a computer-readable storage medium. The storage medium includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc memories, magnetic disk memories, tape memories, or any other medium that can be used to carry or store data and is computer-readable.

[0135] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A distributed optimization fault-tolerant control method for a multi-UAV formation, characterized in that: The following steps are involved: S1, pre-constructing a nonlinear dynamic model of a UAV formation failure, wherein the nonlinear dynamic model includes: Among them, i = 1, 2, ..., N represents the order number of the drones in the formation system, p i (t) = [x i (t),y i (t)] T , v i (t) = [v xi (t),v yi (t)] T , [·] T Represented as the transpose of a matrix, and Respectively represent the first-order derivatives of the position and velocity of the UAV, u i (t) = u si (t)+u oi (t),u si (t) is represented as the sliding mode position control input signal of the UAV in the formation system, u oi (t) is characterized as the optimal control input, u i Represented as the control input of the drones in the current formation, f i1 (p i ,v i ) is characterized as system uncertainty, Characterized by the speed of the drone, p i (t) represents the position of the UAV, D i (t) represents the external disturbance to the UAV, ρ represents the efficiency factor of the actuator, and f is pre-set i1 (p i ,v i ) and D i The value of (t); S2, based on the formation mission requirements, pre-establishes a distributed UAV communication topology structure, using a leader-follower information transmission method; S3, when the UAV formation fails, the current UAV control input u is determined according to the nonlinear dynamic model fi , the position and speed p of the drones in the current formation i =[x i ,y i ] T ,v i =[v xi ,v yi ] T , the trajectory information of the leading UAV in the formation 0i =[x 0i ,y 0i ] T ,v 0i =[v x0i ,v y0i ] T ; S4, respectively determine the global position tracking error z of the i-th UAV in the formation system under the interaction of formation UAVs ip (t) and the global velocity tracking error z iv (t); S5, based on the position information between adjacent UAVs, determine the global tracking error e of the UAV formation i (t); S6, according to the global tracking error, position tracking error and velocity tracking error, the sliding surface s is calculated according to the pre-designed integral sliding mode i (t); S7, calculates the sliding mode position control input u according to the sliding surface, neural network weight adaptive law, and position control input model si (t); S8, calculate the optimal control input u according to the execution network weight adaptive law, the evaluation network weight adaptive law, the global tracking error, and the optimal control input model oi (t); S9, according to the position control input and the optimization control input, the control input of the UAV is adjusted according to the nonlinear dynamics model.

2. The multi-UAV formation distributed optimization fault-tolerant control method according to claim 1 is characterized in that: Step S4 specifically includes the following steps: According to the UAV communication topology, a distributed position and velocity tracking error model of the global system based on the interaction between adjacent UAVs is established in advance; According to the distributed position and velocity tracking error model, the global position tracking error and global velocity tracking error of the i-th UAV in the formation system under the interaction of formation UAVs are calculated and determined respectively. Among them, the distributed position and velocity tracking error model includes: Where i = 1, 2, ..., n, represents the sequence number of the UAVs in the formation system, j represents the UAV adjacent to the i-th UAV, and z ip (t) and z iv (t) are respectively represented as the global position tracking error and global velocity tracking error of the i-th UAV considering the information interaction between the UAVs in the formation, Characterizes the position tracking error of the i-th UAV, It is represented as the position tracking error of the jth UAV adjacent to the i-th UAV in the formation system, p i represents the position of the i-th UAV, p0 represents the trajectory information of the pilot UAV, and h ip The structure characterized by the shape of the formation to be maintained, Characterized as the velocity tracking error of the i-th UAV, v i represents the speed of the ith UAV, v0 represents the trajectory information of the pilot UAV, and h iv The structure characterized by the shape of the formation to be maintained, It is characterized by the velocity tracking error of the jth UAV adjacent to the i-th UAV in the formation system, where i≠j and there is z ip The derivative of is equal to z iv Such a relationship, a ij Represented as an element in the adjacency matrix A, b i Represented as elements in the degree matrix D, the adjacency matrix Degree Matrix 3. The multi-UAV formation distributed optimization fault-tolerant control method according to claim 2 is characterized in that: In step S5, the global tracking error calculation model of the UAV formation is: Among them, e i (t) is characterized as the global tracking error, Characterizes the position tracking error of the i-th UAV, It is represented as the position tracking error of the jth UAV adjacent to the i-th UAV in the formation system, p i represents the position of the i-th UAV, p0 represents the trajectory information of the pilot UAV, Characterized as the velocity tracking error of the i-th UAV, v i represents the speed of the ith UAV, v0 represents the trajectory information of the pilot UAV, and h iv A structure characterized by the shape of the formation to be maintained.

4. The multi-UAV formation distributed optimization fault-tolerant control method according to claim 3 is characterized in that: In step S6, the pre-designed integral sliding mode is: Among them, s i (t) is characterized as the sliding surface, e i (t) is characterized as the global tracking error, e i (0) is characterized by the initial state error, i =[z iv (t),-v0-h iv ] T Characterized as containing z iv (t) and -v0-h iv The matrix, where z iv (t) is characterized by the global velocity tracking error, G i =[0 n*n ,I n ] is represented as a constant matrix, I n Represented as an n-dimensional identity matrix, is an m-dimensional KG i Reversible matrix.

5. According to the multi-UAV formation distributed optimization fault-tolerant control method of claim 4, in step S7, the position control input model is: in, μ i ,k si It is represented as a positive constant gain, sign is represented as a sign function, φ di (x i ) is represented as the selected basis function vector, Represented as the estimated value of the neural network weight, (·) -1 Represented as the inverse matrix of the matrix, s i (t) characterized as sliding surface; The adaptive law of neural network weights is: Among them, k d1 ,k d2 Characterized as a positive constant gain, Characterized as sliding surface s i The transpose of (t), φ di (x i ) is represented as a basis function vector related to the system state.

6. The multi-UAV formation distributed optimization fault-tolerant control method according to claim 5 is characterized in that: Step S8 specifically includes the following steps: The global system tracking error e i (t) is converted into a nominal error system, Among them, U oi =G i u oi It is represented as the control input of the i-th UAV under the nominal error system, According to the optimized nominal error system, the performance index function is established. Among them, J(0) is characterized as a performance index, Characterized as the cost function in the formation system, e i (t) is characterized as the global system tracking error, U oi (t) is represented as the control input of the i-th UAV under the nominal error system, It is represented as the transpose of the control input of the i-th UAV under the nominal error system, is represented as the transpose of the global system tracking error, According to the performance index function, determine the optimal control input under the nominal error system Specifically, in, Characterized as a performance indicator under the nominal system, is represented as the optimal control input, min∫(·) is represented as the minimum value of the performance index, Construct an optimized control input model, specifically: Among them, μ oi Characterized as a normal number to be designed, U oi =G i u oi (t), G i =[0 n*n ,I n ], I n Represented as an n-dimensional identity matrix, Represented as the estimated value of the execution network weight, φ oi (e i ) is represented as a basis function vector related to the global tracking error of the UAV formation; The estimated value of the execution network weight is determined according to the execution network weight adaptation law, which is: Among them, k ai ,k ci Characterized as a positive constant gain, Represented as the estimated value of the evaluation network weight, Characterized by The first time derivative of oi (e i ) is represented as a basis function vector related to the global tracking error of the UAV formation, Represented as the transpose of the basis function vector; The estimated value of the evaluation network weight is determined according to the evaluation network weight adaptive law, which is: Among them, k ci Characterized as a positive constant gain, φ oi (e i ) is represented as a basis function vector related to the global tracking error of the UAV formation, is represented as the transpose of the basis function vector, Characterized by The first time derivative of According to the optimized control input model, iterative convergence is performed to calculate the optimized control input u oi (t).

7. A distributed optimization fault-tolerant control system for a multi-UAV formation, characterized in that: The method is connected to the UAV formation in communication to implement the multi-UAV formation distributed optimization fault-tolerant control method described in any one of claims 1 to 6.

Images