Man-machine cooperation unmanned aerial vehicle optimal encircling control method based on neural network

Abstract

The invention belongs to the technical field of quad-rotor unmanned aerial vehicle cluster control, and particularly relates to a man-machine cooperation unmanned aerial vehicle optimal encircling control method based on a neural network. The method comprises the following steps: establishing a leader dynamic model with commander control input, a position dynamic model of a four-rotor unmanned aerial vehicle follower, an attitude dynamic model and an unmanned cluster directed graph model; and a distributed follower observer, a position loop intelligent learning controller based on an evaluation neural network and a posture loop intelligent learning controller based on the evaluation neural network are designed for each model, so that the optimal encircling control problem of a plurality of leaders under the command of a commander is solved, and the robustness and adaptability are good. In cooperation with a weight parameter updating algorithm based on a dynamic regression extension and hybrid technology, the transient performance of the attitude ring evaluation neural network parameter learning process is improved, and a task is completed under a weak continuous excitation condition.

Description

Technical Field

[0001] The present invention belongs to the technical field of quad-rotor unmanned aerial vehicle cluster control, and specifically is a human-machine collaborative unmanned aerial vehicle optimal encirclement control method based on a neural network. Background Art

[0002] In recent years, with the development of artificial intelligence (AI) technology and the improvement of drone swarm control technology, drone swarm control has been widely used in express logistics, medical search, agricultural production and other fields. UAV encirclement control is an important proposition in drone swarm control. This proposition has applications in space orbiting flight of spacecraft, intelligent group orbiting motion control, unmanned boat formation, sea and air unmanned system coordination, fleet planning and other directions, and has received extensive attention from academia and engineering circles.

[0003] In the encirclement control problem of UAVs, quadcopters are typical underdriven strongly coupled nonlinear dynamic systems, which poses challenges to the optimality and robustness of the encirclement control of swarm systems. In the past, many studies have adopted the method of robust optimal controller to solve this problem, but it can only guarantee the optimality of the nominal system, but not the optimality of the uncertain nonlinear system. Another part of the research has adopted the optimal controller based on reinforcement learning (RL) to solve this problem, but the convergence of its weights depends on the continuous excitation condition, which is difficult to meet in engineering.

[0004] In addition, in many previous studies, the leader of an unmanned swarm usually moves along a fixed path, and the commander is unable to change its movement path in real time according to changes in the task, making it difficult to achieve the purpose of human-machine collaboration. Summary of the invention

[0005] The purpose of the present invention is to provide an optimal encirclement control method for a human-machine collaborative UAV cluster based on adaptive dynamic programming (ADP), which improves the maneuverability of applicable lines and UAV clusters while obtaining the optimality of nonlinear system control.

[0006] To achieve the above object, the present invention adopts the following technical solution:

[0007] A method for optimal encirclement control of a human-machine cooperative UAV based on a neural network comprises the following steps:

[0008] Step 1: Establish a directed graph model of the drone cluster based on the communication connection of the drone cluster. The directed graph model of the drone cluster is used to represent the communication connection between the leader and the follower, and between the followers, including the state information of the leader and the follower; establish the position dynamics model, attitude dynamics model, and leader dynamics model of the quadrotor drone follower. The leader dynamics model is a leader dynamics model with the commander control input; define the follower encirclement error;

[0009] Step 2: Based on the leader dynamics model and the directed graph model of the UAV swarm, a distributed follower observer is designed; the follower observer is used to generate and output the ideal trajectory of the follower that meets the encirclement requirements based on the leader's state information under the condition of unknown commander control input;

[0010] Step 3: Based on the ideal trajectory obtained in step 2 and the position dynamics model of the UAV follower established in step 1, a position loop intelligent learning controller based on an evaluation neural network is designed; the controller learns the optimal control strategy based on the follower state information to generate a virtual control input that enables the follower to track the ideal trajectory;

[0011] Step 4. Based on the virtual control input obtained in step 3 and the attitude dynamics model established in step 1, the ideal attitude of the UAV is solved; according to the ideal attitude of the UAV, an attitude loop intelligent learning controller based on the evaluation neural network is designed, and the dynamic regression extension and mixing (DREM) is used as the weight convergence strategy to generate the actual control input of the UAV under the condition of weak continuous excitation (PE), so that the actual attitude of the UAV can track the ideal attitude and meet the virtual control input output by the position loop controller.

[0012] After adopting the above technical solution, the present invention has the following advantages:

[0013] 1. The present invention takes the dynamic equation of the quadcopter UAV as a dynamic model, which has the characteristics of under-drive, strong coupling and nonlinearity, and designs an optimal position observer and a cascade controller based on a position loop evaluation neural network for this model to control the UAV cluster, so that it can intelligently learn its control strategy through the historical data of the UAV. Compared with the existing multi-agent collaborative control method based on simplified first-order and second-order motion models, the present invention has higher accuracy, and while ensuring the optimality of the nonlinear system, it also improves the robustness.

[0014] 2. The present invention introduces the conductor control command into the leader dynamics model to enable the leader to change the motion path according to the task change during the movement, thus realizing human-machine collaboration and making it more maneuverable. This solves the problem that the leader of the unmanned cluster usually moves along a fixed path, and the conductor cannot change its motion path in real time according to the change of the task, thus making it difficult to achieve human-machine collaboration. In addition, in the case where the introduced conductor control command is unknown to the follower, the leader's trajectory change is learned through the designed cascade controller based on the evaluation network, so that the follower can quickly adapt to the change and still remain in the convex hull formed by the leader.

[0015] 3. The present invention achieves convergence under weak continuous excitation (PE) conditions by adopting a weight update algorithm based on DREM technology in the attitude loop controller; in combination with the scalar regressor convergence method, the transient response performance is improved, making it suitable for the attitude loop controller, a nonlinear system with high response speed requirements. It should be noted that the continuous excitation condition was originally proposed for linear time-invariant systems, requiring that the input signal of the system can fully excite all modes of the system within a sufficiently long time interval so that the observability matrix of the system is full rank. The weak continuous excitation condition is relatively less stringent on the excitation degree of the input signal than the conventional continuous excitation condition, but it still has to ensure that the dynamic characteristics of the system can be continuously stimulated to a certain extent to achieve relevant control objectives. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is the control block diagram of the optimal encirclement control method for a human-machine cooperative quadrotor drone swarm based on adaptive dynamic programming;

[0017] Figure 2 A specific flow chart for implementing the optimal encirclement control method for a human-machine collaborative quadrotor drone swarm based on adaptive dynamic programming. DETAILED DESCRIPTION

[0018] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0019] like Figure 1-Figure 2 As shown, a method for optimal encirclement control of a human-machine cooperative UAV based on a neural network includes the following steps:

[0020] Step 1: Establish a directed graph model of the drone cluster based on the communication connection of the drone cluster, establish the position dynamics model, attitude dynamics model, and leader dynamics model of the quadrotor drone follower, and the leader dynamics model is a leader dynamics model with commander control input; define the follower encirclement error;

[0021] Step 1.1: Establish a directed graph model of the drone cluster to represent the communication between the leader and followers, and between followers. The directed graph model of the drone cluster includes the status information of the leader and the status information of the followers.

[0022] The drone swarm includes x f followers, and x l leaders, and define the follower set as The leader set is

[0023] Define a weighted directed graph G f = {V f ,ε f ,A f} represents the interactive topology of the followers, where V f = {v fi} is the vertex set consisting of followers, represents the set of edges of followers, represents the adjacency matrix of the follower, if (v fi ,v fj )∈ε f , then a ij =1, otherwise a ij =0;

[0024] Assume that the leader has no incoming edges but only outgoing edges, and the follower has both outgoing and incoming edges; based on the assumption, define N fi = {v fj |(v fi ,v fj )∈ε f is the neighbor of the i-th quadrotor drone;

[0025] definition represents the connection matrix between the vth leader and the follower. If the vth leader is connected to the ith follower, then otherwise

[0026] Step 1.2: Establish the position dynamics model and attitude dynamics model of the quadrotor follower:

[0027] The position dynamics model of the quadrotor drone follower is shown in formula (1):

[0028]

[0029] In formula (1), p fi ∈R 3 is the position of the i-th quadrotor drone, For p fi The first-order derivative of For p fi The second derivative of i is the mass of the i-th quadrotor drone, R IB ∈R 3×3 is the transformation matrix from the fuselage coordinate system to the ground coordinate system of the i-th quadrotor drone, I is the ground coordinate system, B is the fuselage coordinate system, and F i ∈R 3 is the total external force on the ith quadrotor. Define ω k,i The angular velocity of the kth (k=1, 2, 3, 4)th rotor of the i-th quadrotor, then the relationship between the resultant external force and the angular velocity of the quadrotor rotor is: in, Represents R IB The transpose of g represents the acceleration due to gravity, c 3,3 ∈R 3 is a column vector with 1 in the third row and 0 in other rows; d pi Represents external disturbance of the position loop;

[0030] The rotational kinematics equation of the quadrotor drone is used as the attitude dynamics model of the quadrotor drone follower. The attitude dynamics model of the quadrotor drone follower is shown in formula (2):

[0031]

[0032] In formula (2), Θ i =(φ i ,θ i ,ψ i ) T ∈R 3 is the attitude angle of the i-th quadrotor drone, φ i represents the roll angle of the i-th quadrotor drone, ψ i represents the pitch angle of the i-th quadrotor, ψ i represents the yaw angle of the i-th quadrotor drone, is θ i The first derivative of is θ i The second derivative of is the inertia matrix of the i-th quadrotor drone, Represents the inertial component of the quadrotor drone in the roll angle direction, Represents the inertial component of the quadrotor drone in the pitch angle direction, Represents the inertial component of the quadrotor drone in the yaw angle direction; is the Coriolis matrix of the i-th quadrotor, τ i ∈R 3 is the external torque, τ i=[τ i,x ,τ i,y ,τ i,z ] T ,T represents transpose, d Θi is the external disturbance of the attitude loop, l ti , k τi With k ωi are the three scaling factors of the i-th quadrotor drone.

[0033] Simplify the position dynamics model and attitude dynamics model of the drone follower:

[0034] Assume that the control input command of the i-th quadrotor drone is: Among them, u zi is the actual position control input of the i-th quadrotor drone, u φi is the actual attitude control input of the i-th quadrotor drone in the roll angle direction, u θi is the actual attitude control input of the i-th quadrotor drone in the pitch angle direction, u ψi is the actual attitude control input of the i-th quadrotor UAV in the yaw angle direction. Through these four control commands, the angular velocities of the four rotors of the quadrotor UAV can be directly calculated, so these four control quantities are defined as the actual control input of the system.

[0035] Based on the underactuated, strongly coupled, and nonlinear characteristics of the quadrotor UAV dynamics equation, it is determined that the relationship between the virtual control input and the actual control input of the system satisfies the formula (3):

[0036]

[0037] In formula (3), u pi ∈R 3 is the virtual control input, φ ri represents the expected roll angle of the i-th quadrotor drone, θ ri The expected pitch angle of the i-th quadrotor drone, ψ ri represents the expected yaw angle of the i-th quadrotor drone.

[0038] Define the position inertia b of the i-th quadrotor drone pi As shown in formula (4):

[0039]

[0040] Substituting formula (3) and formula (4) into formula (1), we can obtain the simplified follower dynamic equation:

[0041]

[0042] Among them, Δ pi is the external disturbance coupling, To control the amount of disturbance coupling;

[0043] Define the attitude inertia b of the i-th quadrotor drone Θi As shown in formula (6):

[0044]

[0045] Substituting formula (6) into formula (2), the simplified follower posture dynamics equation is obtained:

[0046]

[0047] Step 1.3: Establish the leader dynamics model with conductor control input as follows:

[0048]

[0049] Among them, M l The dynamic matrix for leaders, M l ∈R 6×6 ;ζ lv Status information for leaders, T stands for transpose, p lv is the position of the vth leader, is its first-order derivative; define N l =[c 3, 1c 3,1 c 3,1 0 3×3 ],,c 3,1 ∈R 3 is a column vector whose first row is 1 and other rows are 0. From formula (9), we can see that p lv It is the system output of the leader dynamics model;

[0050] Step 1.4: Determine the global encirclement error:

[0051] Assume i The local encirclement error of each follower is e pi ∈R 3 , the calculation formula is as follows:

[0052]

[0053] The local encirclement errors are superimposed to obtain the global encirclement error. Substituting formula (10) into e p get:

[0054]

[0055] In formula (11), the follower state superposition Leader status stacking amount in and Represents the dimension x f ×1 and x l ×1 is a vector whose elements are all 1; and Φ v =(1 / x f )L f +W v , where L f G f The Laplace transform of .

[0056] Step 2: Based on the leader dynamics model and the directed graph model of the UAV cluster, a distributed follower observer is designed; the follower observer generates and outputs an ideal trajectory of the follower that meets the encirclement requirements under the condition of unknown commander control input through the leader's state information; its implementation method includes the following steps:

[0057] Step 2.1, define the estimated encirclement error:

[0058] Suppose the ideal state vector of the i-th follower including position and velocity is Local estimation error in, That is, the ideal position of the i-th follower, is its first-order derivative, T represents transpose;

[0059] Step 2.2: To use the local estimation error of each follower to be close to 0, this embodiment designs a distributed follower observer according to the local estimation encirclement error defined in step 2.1 as follows:

[0060]

[0061] Among them, l i is the conductor control input estimated by the i-th follower observer, where is a normal constant, M l With G l is the cluster leader dynamics equation, S o is the set matrix, is a symbolic function; in order to ensure convergence, the parameters set need to meet the following conditions:

[0062]

[0063] Among them, P>0 and satisfies: is the upper bound of the control input, that is, ||u lv || ∞ Represents u lv The infinite norm of ;

[0064] Using the distributed follower observer designed in step 2.2, the ideal state vector ξ that satisfies the encirclement requirement can be realized under the condition of unknown conductor control input. i Generation; that is, the ideal trajectory generated satisfies the task of making the follower enter the convex hull formed by the leader.

[0065] Step 3: Based on the ideal trajectory obtained in step 2 and the position dynamics model of the UAV follower established in step 1, design a position loop intelligent learning controller based on the evaluation neural network; take the follower state information learning as input, use the controller to learn the optimal control strategy, and thus generate a virtual control input that enables the follower to track the ideal trajectory:

[0066] Step 3.1, write the follower position dynamics equation and the distributed follower observer:

[0067] The simplified UAV follower dynamics equation is rewritten using the state space method, that is, formula (5) is rewritten as follows:

[0068]

[0069] y pi =N pi z pi (17)

[0070] in, is the actual status information, including the actual position and speed of the quadcopter; Yes pi The first derivative of G pi 、M pi are kinetic parameters, D pi =[0I3] T ,M pi =[c 6,4 c 6,5 c 6,6 0 6×3 ] T , and N pi =N l ;y pi is the output of the UAV follower position dynamics equation system, y pi Contains location information, i.e. y pi =p fi .

[0071] Superimposing the state vectors in formula (12), formula (13), formula (16) and formula (17) gives the augmented position system:

[0072]

[0073] Among them, the augmented state variable Z pi The first-order derivative of ; In formula (18), the dynamic parameters of the augmented position system are: T pi =[0G l ] T ,and According to formula (19), the difference between the actual position and the ideal position generated by the observer is obtained

[0074] Step 3.2: Design a cost function for system performance evaluation, use Hamilton function and Bellman optimality principle to solve the cost function, and obtain the optimal control strategy:

[0075] Set the cost function for system performance evaluation:

[0076] For the augmented position system (18)(19), in order to offset the influence of external disturbances, the convergence condition of the position loop controller is designed as follows:

[0077]

[0078] In formula (20), γ p is a constant.

[0079] The cost function designed for performance evaluation of the augmented position system according to the convergence condition of the position loop controller is:

[0080]

[0081] in, T stands for transpose;

[0082] The cost function is solved by using Hamilton function and Bellman optimality principle. By solving the minimum value of the cost function, the augmented position system can meet the convergence condition of the position loop controller, that is, the optimal control strategy of the augmented position system is obtained:

[0083] The Nash condition for the differential game of the position loop controller is defined as:

[0084]

[0085] Solving the Nash conditions of the differential game of the position loop controller, the Hamiltonian equation of the position loop controller is obtained as follows:

[0086]

[0087] in, for The gradient value of represents partial derivative; in order to make the Nash condition of the position loop controller differential strategy hold, it is necessary to let and

[0088] By solving the Hamiltonian equation, we can obtain the optimal position loop controller for the augmented position control system:

[0089]

[0090] Substituting the position loop optimal control strategy formulas (24) and (25) into the position loop controller Hamiltonian equation (23), we obtain:

[0091]

[0092] Step 3.3, design a position loop evaluation neural network, and use the position loop evaluation neural network to approximate the optimal cost function online, so as to obtain the approximate optimal control strategy of the augmented position control system:

[0093] Use the position loop to evaluate the neural network fitting position optimal value function:

[0094]

[0095] Where W cpi is the ideal weight of the neural network for the position loop evaluation, φ pi (δ pi ) is the nonlinear part of the position loop evaluation neural network, ε(δ pi ) is the fitting error.

[0096] Since the ideal weights of the position loop evaluation neural network cannot be obtained, in order to solve this problem, we use As an approximation to the approximation and will The output of the neural network is evaluated as the position loop:

[0097]

[0098] in, is the real weight used for estimation; taking the derivative of the output of the neural network for evaluating the position loop, we get:

[0099]

[0100] for The gradient of , substitute formula (29) into formula (24) and formula (25), and obtain the optimal control law of the approximate position loop as shown in formula (30), and the worst-case interference law of the approximate position loop as shown in formula (31):

[0101]

[0102] in, represents the approximate optimal position loop control quantity, Approximate the worst-case interference of the position loop and design the weight update rate to make Approaching W cpi According to the optimal control law of the position loop and the worst-case interference law, the Replace with Replace with The approximate Hamiltonian is:

[0103]

[0104] In formula (23), represents the approximate Hamiltonian, for The transpose of .

[0105] Step 3.4: Use the gradient descent method as the weight convergence strategy of the position loop evaluation neural network to solve formula (32):

[0106] make So that the Nash condition of the position loop controller differential strategy is established;

[0107] Define the error of the position loop evaluation neural network as Then the error derivative of the position loop evaluation neural network is:

[0108]

[0109] The gradient descent method is used to promote the convergence of the weights of the position loop evaluation neural network so that the objective function Minimum; define the weight convergence strategy of the position loop evaluation neural network as:

[0110]

[0111] where α cpi >0.5, so far we have obtained the position loop controller According to the weight update strategy, the HJB equation can be solved so that the position loop intelligent learning controller can learn the optimal control strategy and output the virtual control input that can make the follower enter the convex hull formed by the leader.

[0112] Step 4: Use the virtual control input obtained in step 3 and the attitude dynamics model established in step 1 to solve the ideal attitude of the UAV; design an attitude loop intelligent learning controller based on the evaluation neural network according to the ideal attitude of the UAV, and use dynamic regression expansion and mixing (DREM) as the weight convergence strategy to generate the actual control input of the UAV under weak continuous excitation (PE) conditions, so that the actual attitude of the UAV tracks the ideal attitude and satisfies the virtual control input output by the position loop controller. The specific implementation method is as follows:

[0113] Step 4.1: Use Newton's method to obtain the ideal posture x of the follower i :

[0114] In the position loop controller: According to formula (30), the approximate optimal control quantity can be obtained According to the approximate optimal control With the current attitude information of the quadrotor drone, use formula (3) to convert u pi Replace with The ideal attitude angle of the quadcopter can be obtained. However, since it is a nonlinear equation system, it is relatively complicated to solve the analytical solution, so this embodiment uses the Newton iteration method to solve the numerical solution of the attitude angle. In addition, to simplify the problem, this embodiment defines the yaw angle ψ ri =0, so we only need to solve the actual position control quantity u zi 、Desired yaw angle φ ri and the desired pitch angle θ ri The specific steps include:

[0115] Let x i =(u zi ,φ ri ,θ ri ) T ∈R 3 is the state vector in Newton iteration method, is the coefficient matrix of Newton's iteration method, then F i (x i )The Jacobi equation is defined as The iterative formula for solving the nonlinear equations using Newton's iteration method is as follows:

[0116]

[0117] According to formula (35), the translational control value u of the following UAV is calculated as zi 、Desired yaw angle φ ri and the desired pitch angle θ ri ; Thus, we can get the ideal posture x of the follower i .

[0118] Step 4.2, write the follower posture dynamics model and the ideal posture dynamics model together:

[0119] In order to control the follower drone to tend to the desired posture, this embodiment adopts a solution method similar to that of the position loop controller. The specific steps are as follows:

[0120] Using the state space method, formula (7) can be rewritten as:

[0121]

[0122] y Θi =N Θi z Θi (37)

[0123] Among them, z Θi Indicates attitude status information. For z Θi The first derivative of y Θi represents the attitude angle; let y Θi is the attitude control system output of formula (36) and formula (37), the attitude dynamics equation parameter G of the follower Θi =[c 6,4 b Θ1,i c 6,5 b Θ2,i c 6,6 b Θ3,i ], D pi =[0 3×3 I3] T , N Θi =[I30 3×3 ] T By Θi The solution is obtained.

[0124] To unify the symbols with the position loop controller, define Δ Θi =d Θi , M Θi (z Θi ) is a nonlinear term and satisfies:

[0125]

[0126] The ideal posture iteratively generated according to formula (35) is expressed by the following ideal posture dynamic equation:

[0127]

[0128] y Θri =N Θri z Θri (40)

[0129] Among them, the ideal posture N Θri =N Θi ,M Θri (z Θri )Define the parameters of the ideal posture dynamics equation: N Θri =N Θi With M Θri (z Θri ), where M Θri (z Θri ) is the unknown equation; y Θri It is the attitude angle information of the system.

[0130] By superimposing formula (36), formula (37) with formula (39) and formula (40), we can obtain the augmented attitude control system, which is expressed as:

[0131]

[0132] in, represents the augmented state variable, the kinematic parameter in the augmented attitude control system Depend on Calculated; δ Θi is the system output of the augmented attitude control system, δ Θi =z Θi -z Θri , calculated by formula (42), is used to represent the difference between the posture and the ideal posture generated in step 4.1.

[0133] Step 4.3: Use the attitude evaluation neural network to approximate the optimal cost function online and obtain the approximate optimal control strategy for the augmented attitude control system:

[0134] In this embodiment, in order to offset the influence of external interference, the convergence condition of the attitude loop controller designed for the augmented attitude control system is:

[0135]

[0136] Among them, γ Θ is a constant.

[0137] In order to meet the convergence conditions, the attitude loop cost function is designed as:

[0138]

[0139] Among them, r Θ (δ Θi ,u Θi ,Δ Θi ) is the local value function of the posture,

[0140] Solving the minimum value of the attitude loop cost function can make the augmented attitude control system meet the convergence condition of the attitude loop controller, that is, obtain the optimal control strategy of the augmented attitude control system. The steps for solving the minimum value of the attitude loop cost function include:

[0141] The Nash condition for the differential game of the attitude loop controller is defined as:

[0142]

[0143] The Nash conditions that satisfy the differential game of the attitude loop controller are calculated, and the attitude loop Hamiltonian equation is obtained:

[0144]

[0145] in, set up and The optimal controller of the attitude loop is solved as:

[0146]

[0147] Step 4.4: Use DREM as the weight convergence strategy of the attitude loop evaluation neural network to solve the attitude loop Hamiltonian (HJB) equation:

[0148] The optimal value function of the attitude loop controller is fitted using the attitude loop neural network as follows:

[0149]

[0150] Among them, W cΘi is the ideal weight of the posture loop evaluation neural network, φ Θi (δ Θi ) is the nonlinear part of the attitude loop evaluation neural network, ε(δ Θi ) is the fitting error; To obtain the ideal weight of the posture loop evaluation neural network, this embodiment uses As an approximation to the approximation and will As the output of the posture loop evaluation neural network:

[0151]

[0152] in, is the true weight used for estimation, T is the transpose;

[0153] Derivative the output of the posture loop evaluation neural network, we get the following formula:

[0154]

[0155] in, for The gradient value of , substitute formula (50) into the approximate value of the attitude loop evaluation neural network, and obtain the solution of formula (47) and the solution of formula (48). The solution of formula 47 is expressed by formula (52), and the solution of formula (48) is expressed by formula (53):

[0156]

[0157] Design the weight update rate to make Approaching W cΘi ,Will Replace with Replace with According to the attitude loop Hamiltonian equation (45), the approximate Hamiltonian is obtained as:

[0158]

[0159] In view of the nonlinear and strong coupling characteristics of the attitude loop, this embodiment adopts the DREM method as a weight convergence strategy to make the weights of the attitude loop neural network converge to the ideal weight value more smoothly under weak PE conditions. The specific steps include:

[0160] According to formula (46), define ∈ HJBΘi The condition that the Hamiltonian function error should satisfy is:

[0161]

[0162] According to formula (55), we can get:

[0163] where Δε T (δ Θi ) is ε(δ Θi ), Δε(δ Θi ) is δ Θi The gradient of T (δ Θi ) is Δε(δ Θi ), which represents the independent variable of the linear equation, represents the dependent variable of the linear equation; therefore, the linear regression equation of the attitude loop is:

[0164]

[0165] Define a first-order filter as:

[0166]

[0167] In formula (57), l cΘi >0 is the forgetting factor;

[0168] Define the filtered regression matrix as:

[0169]

[0170] Substitute formula (57) into formula (58) to obtain the first-order filtered regression matrix as:

[0171]

[0172] According to formula (56) and formula (59), obtain the first-order filtered linear regression formula as:

[0173] Q cΘi (t) = -P cΘi (t)W cΘi -v cΘi (60)

[0174] where, is bounded, i.e., ||v cΘi || < a, representing the filtering Hamiltonian function error; in this embodiment, in order to estimate the unknown parameter W under the relaxed PE condition cΘi , define the mixed matrix formula as:

[0175]

[0176] where, is the adjoint matrix of P cΘi , substitute formula (61) into formula (60) to obtain the mixed linear regression formula as:

[0177]

[0178] where, |P cΘi | is the determinant of P cΘi , then the obtained mixed linear regression scalar formula is:

[0179]

[0180] where, and respectively represent and the j-th row of.

[0181] According to the error drive of the mixed linear regression scalar formula, design the following attitude loop weight convergence strategy:

[0182]

[0183] where, γ2 is a positive constant.

[0184] The HJB equation can be solved by adopting the attitude loop weight convergence strategy expressed in formula (64). On this basis, the attitude loop intelligent learning controller can input the actual control input of the UAV and then change the attitude of the UAV to achieve the ideal attitude to meet the requirements of the virtual control input output by the position loop controller.

Claims

1. A method for optimal encirclement control of a human-machine cooperative UAV based on a neural network, characterized in that: The following steps are involved: Step 1: Establish a directed graph model of the drone cluster based on the communication connection of the drone cluster. The directed graph model of the drone cluster is used to represent the communication connection between the leader and the follower, and between the followers, including the state information of the leader and the follower; establish the position dynamics model, attitude dynamics model, and leader dynamics model of the quadrotor drone follower. The leader dynamics model is a leader dynamics model with the commander control input; define the follower encirclement error; Step 2: Based on the leader dynamics model and the directed graph model of the UAV swarm, a distributed follower observer is designed; the follower observer is used to generate and output the ideal trajectory of the follower that meets the encirclement requirements based on the leader's state information under the condition of unknown commander control input; Step 3: Based on the ideal trajectory obtained in step 2 and the position dynamics model of the UAV follower established in step 1, a position loop intelligent learning controller based on an evaluation neural network is designed; the controller learns the optimal control strategy based on the follower state information to generate a virtual control input that enables the follower to track the ideal trajectory; Step 4. Based on the virtual control input obtained in step 3 and the attitude dynamics model established in step 1, the ideal attitude of the UAV is solved; according to the ideal attitude of the UAV, an attitude loop intelligent learning controller based on the evaluation neural network is designed, and the dynamic regression extension and mixing (DREM) is used as the weight convergence strategy to generate the actual control input of the UAV under weak continuous excitation conditions, so that the actual attitude of the UAV can track the ideal attitude and meet the virtual control input output by the position loop controller.

2. According to the neural network-based optimal encirclement control method for human-machine cooperative UAVs according to claim 1, it is characterized in that: The implementation method of step 1 includes: Establish a directed graph model of drone clusters: The drone swarm includes x f followers, and x l leaders, and define the follower set as The leader set is Define a weighted directed graph G f = {V f ,ε f ,A f } represents the interactive topology of the followers, where V f = {v fi } is the vertex set consisting of followers, represents the set of edges of followers, represents the adjacency matrix of the follower, if (v fi ,v fj )∈ε f , then a ij =1, otherwise a ij =0; Assume that the leader has no incoming edges but only outgoing edges, and the follower has both outgoing and incoming edges; based on the assumption, define N fi = {v fj |(v fi ,v fj )∈ε f is the neighbor of the i-th quadrotor drone; definition represents the connection matrix between the vth leader and the follower. If the vth leader is connected to the ith follower, then otherwise Establish the position dynamics model and attitude dynamics model of the quadrotor drone follower: The position dynamics model of the quadrotor drone follower is shown in formula (1): In formula (1), p fi ∈R 3 is the position of the i-th quadrotor drone, For p fi The first-order derivative of For p fi The second derivative of i is the mass of the i-th quadrotor drone, R IB ∈R 3×3 is the transformation matrix from the fuselage coordinate system to the ground coordinate system of the i-th quadrotor drone, I is the ground coordinate system, B is the fuselage coordinate system, and F i ∈R 3 is the total external force on the i-th quadrotor; define ω k,i The angular velocity of the kth (k=1, 2, 3, 4)th rotor of the i-th quadrotor, then the relationship between the resultant external force and the angular velocity of the quadrotor rotor is: in, Represents R IB The transpose of g represents the acceleration due to gravity, c 3,3 ∈R 3 is a column vector with 1 in the third row and 0 in other rows; d pi Represents external disturbance of the position loop; The attitude dynamics model of the quadrotor UAV follower is shown in formula (2): In formula (2), Θ i =(φ i ,θ i ,ψ i ) T ∈R 3 is the attitude angle of the i-th quadrotor drone, φ i represents the roll angle of the i-th quadrotor drone, ψ i represents the pitch angle of the i-th quadrotor, ψ i represents the yaw angle of the i-th quadrotor drone, is θ i The first derivative of is θ i The second derivative of is the inertia matrix of the i-th quadrotor drone, Represents the inertial component of the quadrotor drone in the roll angle direction, Represents the inertial component of the quadrotor drone in the pitch angle direction, Represents the inertial component of the quadrotor drone in the yaw angle direction; is the Coriolis matrix of the i-th quadrotor, τ i ∈R 3 is the external torque, τ i =[τ i,x ,τ i,y ,τ i,z ] T ,T represents transpose, d Θi represents the external disturbance of the attitude loop, l ti ,k τi With k ωi are the scale factors of the i-th quadrotor drone; Simplify the position dynamics model and attitude dynamics model of the drone follower: Assume that the control input command of the i-th quadrotor drone is: will u zi 、u φi 、u θi , and u ψi Defined as the actual control input; Based on the underactuated, strongly coupled, and nonlinear characteristics of the quadrotor UAV dynamics equation, it is determined that the relationship between the virtual control input and the actual control input of the system satisfies the formula (3): In formula (3), u pi ∈R 3 is the virtual control input, φ ri represents the expected roll angle of the i-th quadrotor drone, θ ri The expected pitch angle of the i-th quadrotor drone, ψ ri represents the expected yaw angle of the i-th quadrotor drone; Define the position inertia b of the i-th quadrotor drone pi As shown in formula (4): Substituting formula (3) and formula (4) into formula (1), we can obtain the simplified follower dynamic equation: Among them, Δ pi is the external disturbance coupling, To control the amount of disturbance coupling; Define the attitude inertia b of the i-th quadrotor drone Θi As shown in formula (6): Substituting formula (6) into formula (2), the simplified follower posture dynamics equation is obtained: The leader dynamics model with conductor control input is as follows: Among them, M l The dynamic matrix for leaders, M l ∈R 6×6 ;ζ lv Status information for leaders, T stands for transpose, p lv is the position of the vth leader, is its first-order derivative; define N l =[c 3,1 c 3,1 c 3,1 0 3×3 ],,c 3,1 ∈R 3 is a column vector with 1 in the first row and 0 in other rows; p lv It is the system output of the leader dynamics model; Determine the global enclosure error: Set up The local encirclement error of each follower is e pi ∈R 3 , and its calculation formula is as follows: The local encirclement errors are superimposed to obtain the global encirclement error. Substituting formula (10) into e p get: In formula (11), the follower state superposition Leader status stacking amount in and Represents the dimension x f ×1 and x l ×1 is a vector whose elements are all 1; and Φ v =(1 / x f )L f +W v , where L f G f The Laplace transform of .

3. The optimal encirclement control method of a human-machine cooperative UAV based on a neural network according to claim 2 is characterized in that: The implementation method of step 2 includes: Step 2.1, define the estimated encirclement error: Let the ideal state vector of the i-th follower including position and velocity be Local estimation error in, That is, the ideal position of the i-th follower, is its first-order derivative, T represents transpose; Step 2.2: Design the distributed follower observer based on the local estimated encirclement error defined in step 2.1 as follows: Among them, l i is the conductor control input estimated by the i-th follower observer, where is a normal constant, M l With G l is the cluster leader dynamics equation, S o is the set matrix, is a sign function; parameter S o , as well as Should meet: Among them, P>0 and satisfies: is the upper bound of the control input, that is, ||u lv || ∞ Represents u lv The infinite norm of ; Step 2.3: Use the distributed follower observer to realize the ideal state vector ξ of the follower that meets the encirclement requirement under the unknown conductor control input i generate.

4. The optimal encirclement control method of a human-machine cooperative UAV based on a neural network according to claim 3 is characterized in that: The implementation method of step 3 includes: Step 3.1, write the follower position dynamics equation and the distributed follower observer: The simplified UAV follower dynamics equation is rewritten using the state space method, that is, formula (5) is rewritten as follows: y pi =N pi z pi (17) in, is the actual status information, Yes pi The first derivative of G pi 、M pi are kinetic parameters, D pi =[0I3] T , M pi =[c 6,4 c 6,5 c 6,6 0 6×3 ] T , and N pi =N l ;y pi is the output of the UAV follower position dynamics equation system, y pi Including location information, i.e. y pi =p fi ; Superimposing the state vectors in formula (12), formula (13), formula (16) and formula (17) gives the augmented position system: Among them, the augmented state variable Z pi The first-order derivative of ; In formula (18), the dynamic parameters of the augmented position system are: T pi =[0G l ] T ,and δ pi Represents the difference between the actual position and the ideal position generated by the follower observer Step 3.2: Design a cost function for augmented position system performance evaluation, use Hamilton function and Bellman optimality principle to solve the cost function, and obtain the optimal control strategy: (1) The convergence condition of the position loop controller designed for the augmented position system is: In formula (20), γ p is a constant; The cost function designed for performance evaluation of the augmented position system according to the convergence condition of the position loop controller is: in, T stands for transpose; (2) The cost function is solved using the Hamilton function and Bellman optimality principle to obtain the optimal control strategy for the augmented position system: The Nash condition for the differential game of the position loop controller is defined as: Solving the Nash conditions of the differential game of the position loop controller, the Hamiltonian equation of the position loop controller is obtained as follows: in, for The gradient value of represents partial derivative; in order to make the Nash condition of the position loop controller differential strategy hold, it is necessary to let and By solving the Hamiltonian equation, we can obtain the optimal position loop controller for the augmented position control system: Substituting the position loop optimal controller equations (24) and (25) into the position loop controller Hamiltonian equation (23), we obtain: Step 3.3, design the position loop evaluation neural network, use the position loop evaluation neural network to approximate the optimal cost function online, and obtain the approximate optimal control strategy of the augmented position control system: Use the position loop to evaluate the neural network fitting position optimal value function: Where W cpi is the ideal weight of the neural network for the position loop evaluation, φ pi (δ pi ) is the nonlinear part of the position loop evaluation neural network, ε(δ pi ) is the fitting error; use As an approximation to the approximation and will The output of the neural network is evaluated as the position loop: in, is the real weight used for estimation; taking the derivative of the output of the neural network for evaluating the position loop, we get: for The gradient of , substitute formula (29) into formula (24) and formula (25), and obtain the optimal control law of the approximate position loop as shown in formula (30), and the worst-case interference law of the approximate position loop as shown in formula (31): in, represents the approximate optimal position loop control quantity, Approximate the worst-case interference of the position loop and design the weight update rate to make Approaching W cpi According to the optimal control law of the position loop and the worst-case interference law, the Replace with Replace with The approximate Hamiltonian is: In formula (23), represents the approximate Hamiltonian, for The transpose of Step 3.4: Use the gradient descent method as the weight convergence strategy of the position loop evaluation neural network to solve formula (32): make So that the Nash condition of the position loop controller differential strategy is established; Define the error of the position loop evaluation neural network as Then the error derivative of the position loop evaluation neural network is: The gradient descent method is used to promote the convergence of the weights of the position loop evaluation neural network so that the objective function Minimum; define the weight convergence strategy of the position loop evaluation neural network as: where α cpi >0.5, the HJB equation is solved according to the position loop evaluation neural network weight convergence strategy to generate a virtual control input that enables the follower to track the ideal trajectory.

5. The optimal encirclement control method of a human-machine cooperative UAV based on a neural network according to claim 4 is characterized in that: The implementation method of step 4 includes: Step 4.1: Use Newton's method to obtain the ideal posture x of the follower i : Define the yaw angle ψ ri =0 Let x i =(u zi ,φ ri ,θ ri ) T ∈R 3 is the state vector in Newton iteration method, is the coefficient matrix of Newton's iteration method, then F i (x i )The Jacobi equation is defined as The iterative formula for solving the nonlinear equations using Newton's iteration method is as follows: According to formula (35), the translational control amount u of the following UAV is calculated: zi 、Desired yaw angle φ ri and the desired pitch angle θ ri ; thus obtaining the ideal posture of the follower; Step 4.2, write the follower posture dynamics model and the ideal posture dynamics model together: Using the state space method, formula (7) can be rewritten as: y Θi =N Θi z Θi (37) Among them, z Θi Indicates attitude status information. For z Θi The first derivative of y Θi represents the attitude angle; let y Θi is the attitude control system output of formula (36) and formula (37), the attitude dynamics equation parameter G of the follower Θi =[c 6,4 b Θ1,i c 6,5 b Θ2,i c 6,6 b Θ3,i ], D pi =[0 3×3 I3] T , N Θi =[I30 3×3 ] T By Θi Solved to get; Definition Δ Θi =d Θi , M Θi (z Θi ) is a nonlinear term and satisfies: The ideal posture iteratively generated according to formula (35) is expressed by the following ideal posture dynamics equation: y Θri =N Θri z Θri (40) Among them, the ideal posture N Θri =N Θi ,M Θri (x Θri );Define the parameters of the ideal posture dynamics equation: N Θri =N Θi With M Θri (z Θri ), where M Θri (z Θri ) is the unknown equation; y Θri is the attitude angle information of the system; By superimposing formula (36), formula (37) with formula (39) and formula (40), we can obtain the augmented attitude control system, which is expressed as: in, represents the augmented state variable, the kinematic parameter in the augmented attitude control system Depend on Calculated; δ Θi is the system output of the augmented attitude control system, δ Θi =z Θi -z Θri , calculated by formula (42), is used to represent the difference between the actual posture and the ideal posture generated in step 4.

1. Step 4.3: Use the attitude evaluation neural network to approximate the optimal cost function online and obtain the approximate optimal control strategy for the augmented attitude control system: The convergence condition of the attitude loop controller of the designed augmented attitude control system is: Among them, γ Θ is a constant; The posture loop cost function designed to meet the convergence conditions is: Among them, r Θ (δ Θi ,u Θi ,Δ Θi ) is the posture value function, By solving the minimum value of the attitude loop cost function, the optimal control strategy of the augmented attitude control system is obtained: The Nash condition for the differential game of the attitude loop controller is defined as: The Nash conditions that satisfy the differential game of the attitude loop controller are calculated, and the attitude loop Hamiltonian equation is obtained: in, set up and The optimal controller of the attitude loop is solved as: Step 4.4: Use dynamic regression extension and hybrid DREM as the weight convergence strategy of the attitude loop evaluation neural network to solve the attitude loop Hamiltonian equation: The optimal value function of the attitude loop controller is fitted using the attitude loop evaluation neural network as shown below: Among them, W cΘi is the ideal weight of the posture loop evaluation neural network, φ Θi (δ Θi ) is the nonlinear part of the attitude loop evaluation neural network, ε(δ Θi ) is the fitting error; use As an approximation to the approximation and will As the output of the posture loop evaluation neural network: in, is the true weight used for estimation, T is the transpose; Derivative the output of the posture loop evaluation neural network, we get the following formula: in, for The gradient value of , substitute formula (50) into the approximate value of the attitude loop evaluation neural network, and obtain the solution of formula (47) and the solution of formula (48). The solution of formula 47 is expressed by formula (52), and the solution of formula (48) is expressed by formula (53): Design the weight update rate to make Approaching W cΘi ;Will Replace with Replace with The approximate Hamiltonian is calculated using formula (45): The DREM method is used as a weight convergence strategy to make the weight of the posture loop neural network converge to the ideal weight value more smoothly under weak PE conditions; the specific steps include: According to formula (46), define ∈ HJBΘi The conditions that the Hamiltonian function error should satisfy are as follows: According to formula (55), we can get: where Δε T (δ Θi ) is ε(δ Θi ), Δε(δ Θi ) is δ Θi The gradient of T (δ Θi ) is Δε(δ Θi ), which represents the independent variable of the linear equation, represents the dependent variable of the linear equation; therefore, the linear regression equation of the attitude loop is: Define a first-order filter as: In formula (57), l cΘi >0 is the forgetting factor; Define the filtered regression matrix as: Substituting formula (57) into formula (58), we get the first-order filter regression matrix: According to formula (56) and formula (59), the first-order filtering linear regression formula is obtained as follows: Q cΘi (t)=-P cΘi (t)W cΘi -v cΘi (60) Among them, is bounded, that is, ||v cΘi || < a, representing the filtering Hamiltonian function error; To estimate the unknown parameter W under the relaxed PE condition cΘi , define the mixing matrix as: in, P cΘi The adjoint matrix of , substituting formula (61) into formula (60), the mixed linear regression formula is obtained as: in, |P cΘi | for P cΘi The determinant of , the obtained mixed linear regression scalar formula is: in, and Respectively represent and The jth row of . According to the hybrid linear regression scalar formula, the following posture ring weight convergence strategy is designed: Among them, γ2 is a positive constant. The attitude loop weight convergence strategy expressed by formula (64) is used to solve the HJB equation. On this basis, the actual control input of the UAV is input through the attitude loop intelligent learning controller, and then the attitude of the UAV is changed to achieve an ideal attitude to meet the requirements of the virtual control input output by the position loop controller.

Images