An Optimal Backstepping Formation Control Method for Multiple Aircraft Based on Azimuth Measurement

Abstract

The present application discloses an optimal backstepping formation control method for multi-aircraft based on azimuth measurement, which is used to solve the optimal control problem of multi-aircraft formation under disturbances. This method first establishes a relative motion dynamics model of the aircraft containing disturbance terms, then constructs formation tracking deviation variables that meet the requirements of aircraft position and azimuth constraints. Next, in the design framework of backstepping control, the optimal performance index function for each step is constructed successively. Finally, a deep reinforcement learning algorithm based on the actor-critic structure is introduced to solve the optimal control law and feed it into the relative motion dynamics model of the aircraft to achieve multi-aircraft formation control. The present invention has good practical significance and application prospects in the field of multi-aircraft cooperative formation control.

Description

Technical Field

[0001] The present application relates to the technical field of multi - vehicle cooperative control. Aiming at the problem of multi - vehicle cooperative formation control under disturbances, an optimal backstepping formation control method for multi - vehicles based on azimuth measurement is designed. Background Art

[0002] With the continuous development of aerospace technology, the level of intelligence and automation of aircraft has been gradually improved, and it has been widely used in military and civilian fields. Compared with a single aircraft, a multi - aircraft system can significantly improve the working efficiency of the system, enhance the task adaptability and safety, and reduce the task risk cost through distributed cooperative operations. Therefore, multi - vehicle cooperative control technology has received more and more attention from researchers. However, the complex and changeable flight environment inevitably leads to external interference on the multi - vehicle system during cooperative task execution, thus affecting the system stability. In addition, in existing research, the measurement and transmission of position and velocity information are inevitable for multi - vehicle cooperative formation control, but in practice, the true position and velocity information of multi - vehicles is not always available in real - time. On the other hand, considering optimization control problems such as optimal fuel consumption and shortest time of aircraft is also a very meaningful research at present, which reflects the inevitable requirement for the development of the system's ordered structure to a higher level. Therefore, in order to improve the safety level of overall operations and maximize the economic benefits of multi - vehicle cooperative operations, the present application studies the optimal backstepping formation control technology for multi - vehicles based on azimuth measurement. Summary of the Invention

[0003] Object of the Invention

[0004] Aiming at the problems of external disturbances and optimal control encountered by the multi - vehicle system during cooperative task execution, the purpose of the present application is to design an optimal backstepping formation control method for multi - vehicles based on azimuth measurement, to ensure that the multi - vehicle system can still operate stably under disturbances, and to achieve optimal tracking control under the desired formation flight trajectory and azimuth angle constraints.

[0005] Technical Solution

[0006] To achieve the above object, the present application adopts the following technical solutions:

[0007] An optimal backstepping formation control method for multi - vehicles based on azimuth measurement, the method comprising the following steps:

[0008] Step 1: Establish a relative - motion dynamics model of the multi - vehicle formation system considering the influence of external disturbances, and transform the relative - motion dynamics model of the aircraft;

[0009] Step 2: Construct tracking deviation variables that meet the requirements of multi - vehicle formation position and azimuth angle constraints;

[0010] Step 3: Define the optimal performance index function, use backstepping control to design the intermediate virtual control law and the real control law, and then solve the Bellman equation based on the deep reinforcement learning algorithm with an actor-critic structure to obtain the approximate optimal values of the virtual control law and the real control law;

[0011] Step 4: Feed the designed backstepping formation control law into the relative motion dynamics model of the aircraft to achieve optimal formation control of multiple aircraft under disturbances.

[0012] Furthermore, the specific process of Step 1 is as follows:

[0013] Step 1.1: For the formation motion system containing N aircraft, establish the relative motion dynamics model of the i-th aircraft:

[0014]

[0015] where p i = [x i , y i , z i T is the relative position of the i-th aircraft, represents the first derivative of p i , represents the second derivative of p i , m i is the mass of the i-th aircraft, u i is the control input, d i is the orbital disturbance term, C i represents the skew-symmetric matrix similar to Coriolis, D i represents the time-varying potential energy matrix, n i represents the time-varying gravitational vector, and the specific expressions of C i , D i and n i are as follows:

[0016]

[0017] where θ is the true anomaly of the leader aircraft, satisfying and e c , a c and are the orbital eccentricity, semi-major axis, and mean orbital angular velocity of the leader aircraft respectively; μ is the gravitational coefficient of the earth, is the distance of the i-th aircraft from the center of the earth, is the distance between the center of the earth's mass and the leader aircraft; The above formula (1) is transformed as follows to obtain:​

[0018]

[0019] Among them, is a non - linear term, g i =-1 / m i I3 is a gain term, I3 represents a 3×3 identity matrix, d i1 =1 / m i d i represents a lumped disturbance term.

[0020] Furthermore, the specific process of the second step includes the following:

[0021] Define the tracking deviation variable δ i1 as:

[0022]

[0023] Among them, represents the desired relative position vector of the i - th aircraft, λ1 is a control parameter set by the user, represents the desired azimuth configuration between the i - th aircraft and the j - th aircraft, b ij is the azimuth information of the i - th aircraft and the j - th aircraft obtained by actual measurement; a ij is an identifier indicating whether the azimuth information between the i - th aircraft and the j - th aircraft is measurable.

[0024] Furthermore, the specific process of the third step includes the following:

[0025] Step 4.1, define the following optimal performance index function:

[0026]

[0027] Among them, Ω1 is a compact set containing the origin, is a cost function, α i1 is a virtual control signal, is the optimal virtual control signal; by calculating the derivative of the above formula (4), the Bellman equation of the tracking deviation variable δ i1 is:

[0028]

[0029] By solving the optimal virtual control signal is obtained as:

[0030]

[0031] Among them, represents with respect to δi1 The partial derivative of

[0032]

[0033] where β1 is a control parameter set by the user, is an unknown term related to the optimal performance index function and the tracking deviation variable; substituting equation (7) into equation (6) above gives the optimal virtual control signal:

[0034]

[0035] Using a neural network to approximate in equation (8) above, we get:

[0036]

[0037] where represents the ideal weight vector, S i1 (δ i1 ) is a Gaussian basis function vector, ε i1 (δ i1 ) represents the approximation error, bounded by a constant ε i1 max , i.e., satisfying |ε i1 (δ i1 )| ≤ ε i1 max ; using equation (9) above, the optimal virtual control signal is rewritten as

[0038]

[0039] Using a deep reinforcement learning algorithm based on the actor-critic structure to solve the Bellman equation of subsystem (3), so we have:

[0040]

[0041]

[0042] where and are the estimates of and respectively, and are the weight vectors of the actor-critic neural network, and the weights of the actor-critic neural network are updated by the following equation:

[0043]

[0044]

[0045] where γ i1c and γ i1ais a control parameter set by the user;

[0046] Step 4.2, define the following optimal performance index function:

[0047]

[0048] where, is the cost function, is the speed error signal, u i is the control input signal, is the optimal control signal; the Bellman equation of the optimal performance index function is calculated as:

[0049]

[0050] By solving the optimal control signal is obtained as:

[0051]

[0052] where, denotes the partial derivative of δ i2 and can be decomposed as:

[0053]

[0054] where, β2 is a control parameter set by the user, is the unknown term. By substituting Equation (18) into Equation (17), we get

[0055]

[0056] Since the in Equation (19) is unknown but continuous on the compact set Ω2, a neural network is used to approximate and we get:

[0057]

[0058] where, denotes the ideal weight vector, S i2 (δ i2 ) is the Gaussian basis function vector, ε i2 (δ i ) represents the approximation error, bounded by a constant ε i2 max , that is, satisfying |ε i2 (δ i2 )| ≤ ε i2 max ;

[0059] Using the deep reinforcement learning algorithm based on the actor-critic structure to solve the Bellman equation (16), we get:

[0060]

[0061]

[0062] Among them, and are respectively and estimates, and are the weight vectors of the neural network, which are updated by the following formula:

[0063]

[0064]

[0065] Among them, γ i2c and γ i2a are control parameters set by the user.

[0066] Furthermore, the specific process of the fourth step includes the following:

[0067] Send the designed optimal backstepping formation control law into the relative motion dynamics model of the aircraft affected by the disturbance term d i to obtain the formation position of the i-th aircraft, thereby realizing the optimal formation control of multiple aircraft under disturbance.

[0068] As a preferred implementation of this application, the selection of the above control parameters β2, γ i2c and γ i2a satisfies:

[0069]

[0070] Beneficial effects:

[0071] The beneficial technical effects of this application are:

[0072] (1) This application discloses an optimal backstepping formation control method for multiple aircraft based on azimuth measurement. This method considers the cooperative formation control problem of a multiple aircraft system affected by external disturbances. The designed control scheme can ensure the stable operation of the multiple aircraft system under disturbances and achieve optimal tracking control under formation position and azimuth constraints.

[0073] (2) This application considers the cooperative formation problem of multiple aircraft when inter-agent communication cannot be carried out between them, that is, when the distance or position information of the formation members cannot be transmitted, and each aircraft can only measure the azimuth angle information between it and adjacent aircraft, and realizes the formation control of multiple aircraft based on azimuth measurement.

[0074] (3) The present application introduces a deep reinforcement learning algorithm based on the actor-critic structure to solve the Bellman equation and embeds it into the design architecture of backstepping control to obtain the optimal formation control law for the multi-aircraft system. Description of the Drawings

[0075] Figure 1 Flowchart of the optimal backstepping formation control scheme for the multi-aircraft system under disturbance;

[0076] Figure 2 System block diagram of the optimal backstepping formation control scheme for the multi-aircraft system under disturbance;

[0077] Figure 3 Interaction topology structure diagram of the multi-aircraft system;

[0078] Figure 4 Formation operation trajectory diagram of the multi-aircraft system;

[0079] Figure 5 Formation tracking error δ of the multi-aircraft system i1 Curve graph;

[0080] Figure 6 Control input of the multi-aircraft system Curve graph. Detailed Implementation Manner

[0081] In combination with the attached drawings, the control method of the present application will be further explained.

[0082] (a) Establish a relative motion dynamics model of the aircraft considering the influence of external disturbances:

[0083]

[0084] Among them, i represents the serial number of the aircraft, p i = [x i , y i , z i T represents the relative position vector of the aircraft, represents the first derivative of p i , represents the second derivative of p i , m i represents the mass of the aircraft i, u i represents the control input of the aircraft i, d i represents the orbital disturbance term of the aircraft i. C i , D i and n i are given as:

[0085]

[0086] where θ is the true anomaly of the leader aircraft, satisfying and e c , a c and represent the orbital eccentricity, semi-major axis, and mean orbital angular velocity of the leader aircraft. μ is the Earth's gravitational coefficient, is the distance of aircraft i from the Earth's center, is the distance between the Earth's mass center and the leader aircraft. Making the following changes to equation (1) above, we get:

[0087]

[0088] where g i = -1 / m i I3 and I3 represents the 3×3 identity matrix, d i1 = 1 / m i d i .

[0089] (b) To meet the requirements of aircraft formation position and orientation constraints, the following tracking deviation variables are constructed:

[0090]

[0091] where represents the desired relative position vector of aircraft i, represents the desired azimuth angle configuration between aircraft i and aircraft j, b ij is the azimuth angle information measured between aircraft i and aircraft j. Here, a ij = 1 only indicates that the azimuth angle information between aircraft i and aircraft j is measurable, and a ij = 0 indicates that the azimuth angle information between the two is not measurable.

[0092] (c) Based on the relative motion dynamics model of the multi-aircraft formation system with external disturbances constructed in step one, using the tracking deviation variables defined in step two to define the optimal performance index, and combining the backstepping method with the deep reinforcement learning algorithm based on the actor-critic structure to design the optimal formation controller. The specific steps are as follows:

[0093] Step one: Define the following optimal performance index function:

[0094]

[0095] where Ω1 is a compact set containing the origin, is the cost function, α i1 is the virtual control signal, is the optimal virtual control signal, and there is By calculating the derivative of Equation (4), δ i1 The Bellman equation of the subsystem can be written as:

[0096]

[0097] Solve The obtained optimal virtual control signal is

[0098]

[0099] where can be decomposed into:

[0100]

[0101] By substituting the above Equation (7) into Equation (6), it can be obtained that

[0102]

[0103] Since the in the above Equation (8) is unknown but continuous on the compact set Ω1, the following neural network can be used to approximate it

[0104]

[0105] where represents the ideal weight vector, S i1 (δ i1 ) is the Gaussian basis function vector, ε i1 (δ i1 ) represents the approximation error, satisfying |ε i1 (δ i1 )| ≤ ε i1max . Using the above Equation (9), the optimal virtual control signal can be rewritten as

[0106]

[0107] However, the above optimal virtual control signal is not available because the ideal weight signal is unknown. In order to obtain an available virtual control signal, the deep reinforcement learning algorithm based on the actor-critic structure is used to solve the Bellman equation of δ i1 subsystem, and it can be obtained that

[0108]

[0109]

[0110] where the weight vectors and Updated by the following formula:

[0111]

[0112]

[0113] The above control parameters β1, γ i1c and γ i1a are selected to satisfy:

[0114]

[0115] Next, the derivation details of the actor-critic neural network weight update law are given. First, the Bellman residual e i1 is defined as:

[0116]

[0117] where, is the approximation of the Bellman equation. According to the previous analysis, the optimal solution satisfies e i1 → 0, that is is maintained, which is equivalent to:

[0118]

[0119] To derive the reinforcement learning weight update law that can ensure the minimum of equation (17), the following positive definite function

[0120]

[0121] Obviously, F i1 = 0 is equivalent to equation (17). The weight update laws (13)-(14) are derived based on the following facts. First, along equations (13)-(14), the above formula is differentiated to obtain:

[0122]

[0123] where,

[0124] Let Then the derivative of δ i1 can be rewritten as:

[0125]

[0126] Consider the following candidate Lyapunov function for the δ i1 subsystem:

[0127]

[0128] where, They are the estimation errors of the weights of the actor-critic neural network respectively. Taking the derivative of the above formula along formulas (20), (12), (13), and (14), we can obtain:

[0129]

[0130] According to Young's inequality, we can get:

[0131]

[0132] Substituting formula (23) into (22), we can get:

[0133]

[0134] In addition, according to we can obtain:

[0135]

[0136]

[0137] Substituting formulas (25)-(26) into (24), we can get:

[0138]

[0139] where is bounded by the constant c i1max Using Young's inequality, we can obtain:

[0140]

[0141] Using condition (15), formula (27) can be rewritten as:

[0142]

[0143] Let denote the minimum eigenvalue of, then we have:

[0144]

[0145] Substituting formula (30) into formula (29), the following inequality holds:

[0146]

[0147] where

[0148] Step 2: Define the following optimal performance index function:

[0149]

[0150] where, is the cost function, u i is the control signal, is the optimal control signal. By calculating the derivative of the above formula, δ can be obtained i2 The Bellman equation of the subsystem is:

[0151]

[0152] By solving the optimal control signal can be obtained as:

[0153]

[0154] where, can be decomposed into:

[0155]

[0156] Substituting Equation (34) into Equation (33), we get

[0157]

[0158] Since in the above formula (35) is unknown but continuous on the compact set Ω2, a neural network can be used to approximate it, resulting in:

[0159]

[0160] where, represents the ideal weight vector, S i2 (δ i2 ) is the Gaussian basis function vector, ε i2 (δ i ) represents the approximation error, satisfying |ε i2 (δ i2 )| ≤ ε i2 max . Similarly, in order to obtain an available control signal, the Bellman equation of the δ i2 subsystem is solved using a deep reinforcement learning algorithm based on the actor-critic structure. Therefore, we have:

[0161]

[0162]

[0163] where, and are the weight vectors of the neural network, which are updated by the following formula:

[0164]

[0165]

[0166] The above control parameters β2, γ i2c and γ i2a are selected to satisfy:

[0167]

[0168] Consider the following Lyapunov function candidate for the subsystem δ i2 :

[0169]

[0170] where, are the estimation errors of the weights of the actor-critic neural network, respectively. Similar to the first-step derivation, the derivative of L i2 can be written as:

[0171]

[0172] where, is bounded by a constant c i2 max . Therefore, there exists the following fact:

[0173]

[0174] From the above equation (45), it can be seen that the error signals δ i1 , δ i2 , and are semi-globally uniformly ultimately bounded.

[0175] Therefore, the stability of the multi-aircraft formation system is proven by the above equation, which can not only meet the requirements of formation position and orientation constraints, but also all error signals are semi-globally uniformly ultimately bounded.

[0176] (e) Finally, feed the above-designed optimal control input signal into the relative motion dynamics model of the aircraft to achieve the optimal formation control of the multi-aircraft system under external disturbances.

[0177] The effectiveness of the present invention application is verified by the following simulation:

[0178] The interaction topology of four aircraft is as Figure 3 shown. In the simulation, the parameters of the aircraft are set as: μ = 3.986×10 14 m 3 / s 2 , θ = 0 rad, a c = 6.878×10 7 km, e c = 0.01, m i= 1 kg, i = 1, 2, 3, 4. The disturbance term is set to d i = 10 -2 × [1.2sin(0.003t); 0.5sin(0.005t); -sin(0.004t)]; The initial states of the formation aircraft are set to p1 = [-35; -19.5; 1]; p2 = [35; -29; 1.5]; p3 = [25; 28; 0.2]; p4 = [-33.5; 30; 3.7]; The desired positions of the formation aircraft are set to The control parameters are selected as λ1 = 5, β1 = 1.1, β2 = 1.1, γ i1a = 10, γ i1c = 6, γ i2a = 100, γ i2c = 60.

[0179] The formation flight trajectories of the multi-aircraft system are as Figure 4 shown. It can be seen from the figure that by using the proposed optimal backstepping formation control scheme, the four aircraft achieve the desired square formation configuration under disturbance conditions. Figure 5 The formation tracking error of the multi-aircraft system is provided. Figure 6 The control signals of the multi-aircraft system are provided. It can be seen from the simulation results that the formation tracking error of the multi-aircraft system is convergent, even in the presence of disturbances.

[0180] In summary, the control method designed by the present invention can effectively perform cooperative formation control on the multi-aircraft system, even in the presence of external disturbances.

Claims

1. A multi-aircraft optimal backstepping formation control method based on azimuth measurement, characterized in that: The method comprises the following steps: Step 1: establishing a relative motion dynamics model of a multi-aircraft formation system taking into account the influence of external disturbances, and transforming the relative motion dynamics model of the aircraft; Step 2: construct a tracking deviation variable that meets the position and azimuth constraints of the multi-aircraft formation; the step 2 specifically includes the following process: The tracking deviation variable δ i1 Defined as: in, represents the desired relative position vector of the i-th aircraft, λ1 is the control parameter set by the user, represents the desired azimuth configuration between the i-th aircraft and the j-th aircraft, b ij is the azimuth angle information between the i-th aircraft and the j-th aircraft obtained by actual measurement; a ij An identifier indicating whether the azimuth information between the i-th aircraft and the j-th aircraft is measurable; Step 3: Define the optimal performance index function according to the tracking deviation variable, use backstepping control to design the intermediate virtual control law and the real control law, and then solve the Bellman equation based on the deep reinforcement learning algorithm of the actor-critic structure to obtain the approximate optimal values ​​of the virtual control law and the real control law; Step 4: Input the designed backstepping formation control law into the relative motion dynamics model of the aircraft to achieve optimal formation control of multiple aircraft under disturbance conditions.

2. The multi-aircraft optimal backstepping formation control method based on azimuth measurement according to claim 1 is characterized in that: The step 1 specifically includes the following process: Step 1.1, for the formation motion system consisting of N aircraft, establish the relative motion dynamics model of the i-th aircraft: Among them, p i =[x i ,y i ,z i ] T is the relative position of the i-th aircraft, Indicates p i The first derivative of Indicates p i The second derivative of i is the mass of the i-th aircraft, u i is the control input, d i is the orbital perturbation term, C i represents a skew-symmetric Coriolis-like matrix, D i represents the time-varying capability matrix, n i represents the time-varying gravitational vector, C i , D i and n i The specific expression is as follows: Where θ is the true approach angle of the leader vehicle, satisfying and e c 、a c and are the orbital eccentricity, semi-major axis and mean orbital angular velocity of the leader spacecraft; μ is the gravitational coefficient of the Earth, is the distance of the ith aircraft relative to the center of the earth, is the distance from the center of mass of the Earth to the leader spacecraft; the above formula (1) is changed as follows to obtain: in, is a nonlinear term, g i =-1 / m i I3 is the gain term, I3 represents the 3×3 unit matrix, d i1 =1 / m i d i represents the lumped disturbance term.

3. The optimal backstepping formation control method for multiple aircraft based on azimuth measurement according to claim 1 is characterized in that: The step three specifically includes the following process: Step 4.1, define the following optimal performance indicator function: Among them, Ω1 is a compact set containing the origin, is the cost function, α i1 is the virtual control signal, is the optimal virtual control signal; by calculating the derivative of the above formula (4), the tracking deviation variable δ i1 The Bellman equation is: By solving The optimal virtual control signal is: in, express δ i1 The partial derivative of is decomposed into: Among them, β1 is the control parameter set by the user, is the unknown term related to the optimal performance index function and the tracking deviation variable; by substituting the above formula (7) into the above formula (6), the optimal virtual control signal is obtained: Using neural network to solve the above equation (8) Approximately, we get: in, represents the ideal weight vector, S i1 (δ i1 ) is the Gaussian basis function vector, ε i1 (δ i1 ) represents the approximation error, with a constant ε i1max is bounded, that is, it satisfies |ε i1 (δ i1 )|≤ε i1max ; Using the above formula (9), the optimal virtual control signal is rewritten as, Solve δ using a deep reinforcement learning algorithm based on the actor-critic structure i1 The Bellman equations of the subsystem thus have: in, and They are and The estimate, and are the weight vectors of the actor-critic neural network respectively, and the weights of the actor-critic neural network are updated as follows: Among them, γ i1c and γ i1a are control parameters set by the user; Step 4.2, define the following optimal performance indicator function: in, is the cost function, is the speed error signal, u i To control the input signal, is the optimal control signal; the Bellman equation of the optimal performance index function is calculated as: By solving The optimal control signal is: in, express δ i2 The partial derivative of can be decomposed into: Among them, β2 is the control parameter set by the user, is an unknown term; by substituting formula (18) into formula (17), we can get: Since in formula (19) Unknown but continuous on the compact set Ω2, using a neural network Approximately, we get: in, represents the ideal weight vector, S i2 (δ i2 ) is the Gaussian basis function vector, ε i2 (δ i2 ) represents the approximation error, with a constant ε i2max is bounded, that is, it satisfies |ε i2 (δ i2 )|≤ε i2max ; Using the deep reinforcement learning algorithm based on the actor-critic structure to solve the Bellman equation (16), we get: in, and They are and The estimate, and is the weight vector of the neural network, which is updated by the following formula: Among them, γ i2c and γ i2a It is a control parameter set by the user.

4. The multi-aircraft optimal backstepping formation control method based on azimuth measurement according to claim 1 is characterized in that: The step 4 specifically includes the following process: The optimal backstepping formation control law designed Enter the disturbance term d i In the relative motion dynamics model of the affected aircraft, the formation position of the i-th aircraft is obtained, thereby realizing the optimal formation control of multiple aircraft under disturbance conditions.

Images