Aircraft self-learning optimization control method based on adaptive dynamic programming

Abstract

The invention discloses an aircraft self-learning optimization control method based on adaptive dynamic programming, and the method comprises the steps: firstly, designing a basic stability controller under a backstepping method frame in order to guarantee the stable control of an aircraft under the uncertain and unknown external disturbance of a model; then, considering that the tracking performance of the system has a weakening trend along with disturbance change, introducing an ADP-based parameter self-adjustment intelligent optimization algorithm to design an auxiliary controller, and obtaining a compensation control signal by executing and evaluating updating adjustment of a network weight along with tracking deviation so as to further reduce the tracking error; and the self-learning and self-adaptive capabilities of the controller to environmental changes are realized. And finally, integrating the basic controller and the auxiliary optimization controller to finally obtain a complete robust self-learning optimization controller. The method is high in universality and high in calculation precision, and has important engineering practice reference significance.

Description

Technical Field

[0001] The present invention belongs to the technical field of aircraft, and in particular relates to an aircraft self-learning optimization control method based on adaptive dynamic programming. Background Art

[0002] Flight control system is a key technology to ensure the safe and reliable flight of aircraft. At present, the commonly used flight control methods include classical PID control, LQR control, sliding mode control, dynamic surface control, backstepping control, neural network control, etc. These control methods are all based on a stable controller designed by Lyapunov stability, which realizes system stability, adaptability and robustness under the conditions of model uncertainty, external disturbances, actuator failures, etc. However, none of these control methods completely solves the general adaptive control problem of aircraft under parameter uncertainty and environmental disturbances. In other words, most control methods are usually predefined controllers designed on fixed parameters and known system models. The controller parameters cannot be self-learned with changes in environmental disturbances and uncertainties, which affects the control performance of the control system. Summary of the invention

[0003] In order to overcome the shortcomings of the prior art, the present invention provides an aircraft self-learning optimization control method based on adaptive dynamic programming. First, in order to ensure the stable control of the aircraft under model uncertainty and unknown external disturbances, a basic stability controller is designed under the framework of backstepping. Then, considering that the system tracking performance will show a weakening trend with the change of disturbance, an auxiliary controller is designed based on the parameter self-adjustment intelligent optimization algorithm of ADP. The compensation control signal is obtained by adjusting the execution-evaluation network weights with the update of the tracking deviation to further reduce the tracking error, and realize the self-learning and adaptive ability of the controller to environmental changes. Finally, by integrating the basic controller and the auxiliary optimization controller, a complete robust self-learning optimization controller is finally obtained. The controller can ensure that the system control performance can achieve rapid convergence under uncertainty and environmental disturbances, and achieve optimality under given performance indicators. The method has strong versatility and high calculation accuracy, and has important reference significance for engineering practice.

[0004] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0005] Step 1: Model building;

[0006] Consider the following strict feedback system:

[0007]

[0008] in, u∈R represents the system input, y∈R represents the system output, Indicates about An unknown smooth function of represents a known non-zero function, d i (t) represents the time-varying disturbance, satisfying in and Represents the upper bound of disturbance and disturbance change rate;

[0009] Step 2: Basic controller design based on backstepping method;

[0010] Step 2-1: According to formula (1), define the tracking error as e1 = yy r , where y r Represents reference signal; designs virtual control quantity for:

[0011]

[0012] in, represents the adaptive parameter estimates associated with the optimal weights of the neural network, represents the neural network basis function vector, k1>0, λ1>0 is a designed normal constant;

[0013] Step 2-2: Define the tracking error as Design virtual control volume for:

[0014]

[0015] in, Represents the neural network basis function vector;

[0016] Step 2-3: Define the tracking error as The actual control quantity u is designed as:

[0017]

[0018] in, Represents the neural network basis function vector;

[0019] Design parameters The adaptive law is:

[0020]

[0021] Among them, σ>0 and η>0 are designed normal constants;

[0022] Step 3: Design of auxiliary controller based on ADP;

[0023] Step 3-1: Return the controller u designed in step 2 to system (1) to obtain the output prediction value y pre , define the prediction tracking error as epre =y pre -y r ;

[0024] Step 3-2: Design the evaluation network as follows:

[0025]

[0026] Where z(t)=[e pre (t-1),e pre (t)] T represents the forecast tracking error, u a (t)=[u a1 (t),…,u am (t)] T represents the auxiliary control input, p cj (t) and q cj (t) represents the input and output of the jth hidden layer node of the evaluation network, represents the weight from the i-th input layer to the j-th hidden layer of the evaluation network, represents the weight between the hidden and output layers of the evaluation network, is the estimated value of the cost function J(t), Where 0<β≤1 represents the discount factor, r(t)=z(t) T Qz(t)+u a (t) T Ru a (t) represents the utility function, Q∈R 2×2 , R∈R m×m is the designed positive definite symmetric matrix;

[0027] Step 3-3: Design the execution network as follows:

[0028]

[0029] Where: p aj (t) and q aj (t) represents the input and output of the jth hidden layer node of the execution network, u ak (t) represents the kth control variable output by the execution network, represents the weight of executing the network's ith input to the jth hidden layer, Represents the weights from the kth hidden layer to the jth output layer of the execution network;

[0030] Step 3-4: Design the weight update law of the evaluation network as:

[0031] w c (t+1)=w c (t)-μ c βec (t)φ cj (t) (8)

[0032] Where: μ c >0 is the designed evaluation network learning rate;

[0033] Step 3-5: Design and execute the network weight update law as follows:

[0034]

[0035] Among them: The element in row i and column k of matrix C(t) is μ a >0 is the designed execution network learning rate;

[0036] Step 4: Integrate the basic controller and the auxiliary controller to control the system;

[0037] According to the control input u obtained in step 2 and the control input u obtained in step 3 a The total control input U = u + u a , the control input U is returned to the flight dynamics model equation (1) to achieve system command tracking control.

[0038] Preferably, in the aircraft self-learning optimization control method based on adaptive dynamic programming, the aircraft longitudinal dynamics model containing unknown time-varying interference is constructed as follows:

[0039]

[0040] Where V, γ, α and q represent velocity, track angle, angle of attack and pitch rate respectively; T, L and M yy Respectively represent engine thrust, lift and pitch moment; m, I yy and g represent the mass of the aircraft, the moment of inertia of the y-axis, and the acceleration due to gravity, respectively; δ e is the elevator deflection angle; d γ =0.001sin(0.2t),d α =0.003cos(0.1t),d q =0.001sin(0.3t) represents the interference in the three subsystems respectively.

[0041] The beneficial effects of the present invention are as follows:

[0042] The present invention treats the model uncertainty and external interference in the flight process of the aircraft as a state-related nonlinear function, and uses a neural network for preliminary estimation and compensation in the design of the basic controller. On this basis, a parameter self-adjustment intelligent optimization algorithm based on ADP is further introduced. The compensation control signal is obtained by updating the execution-evaluation network weights with the tracking deviation to further suppress uncertainty and disturbance, thereby reducing the tracking error and improving the tracking performance. This method can be widely used in the design of various aircraft flight control systems with high requirements for trajectory tracking accuracy, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0044] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0045] The purpose of the present invention is to provide an aircraft self-learning optimization control method based on ADP to solve the problem of self-optimization stable tracking control of the aircraft flight control system under model uncertainty and external disturbance. In order to integrate learning ability and adaptive ability into the design of flight control system to suppress the influence of parameter uncertainty and environmental uncertainty on control performance, a robust self-learning optimization control method based on ADP is proposed.

[0046] The present invention discloses a self-learning optimization control method for an aircraft based on adaptive dynamic programming. The innovation of this method lies in that, on the basis of a flight controller designed by a backstepping control strategy, a parameter self-adjusting intelligent optimization algorithm based on adaptive dynamic programming (ADP) is introduced to design an auxiliary controller, and the tracking accuracy is further improved by obtaining a compensation control signal by updating and adjusting the execution-evaluation network weights along with the tracking deviation, thereby realizing the self-learning and self-adaptation capabilities of the controller to environmental changes.

[0047] The present invention first establishes the longitudinal dynamics model of the aircraft; then designs a basic controller that can ensure flight stability under the framework of the backstepping method; then introduces an ADP-based parameter self-adjusting intelligent optimization algorithm to design an auxiliary controller, and obtains a compensation control signal by updating and adjusting the execution-evaluation network weights with the tracking deviation to further reduce the tracking error, thereby realizing the controller's self-learning and self-adaptation capabilities to environmental changes; finally, by integrating the basic controller and the auxiliary optimization controller, a complete robust self-learning optimization controller is finally obtained. The algorithm logic block diagram is shown below. Figure 1 This method has a simple structure, is easy to implement, has strong engineering practice ability, and can be widely used in the design of flight control systems for various aircraft.

[0048] The design steps are described as follows:

[0049] Step 1: Model building;

[0050] Consider the following strict feedback system:

[0051]

[0052] in, u∈R represents the system input, y∈R represents the system output, Indicates about An unknown smooth function of represents a known non-zero function, d i (t) represents the time-varying disturbance, satisfying in and Represents the upper bound of disturbance and disturbance change rate.

[0053] Step 2: Basic controller design based on backstepping method;

[0054] (a) According to formula (1), the tracking error is defined as e1 = yy r , where y r Represents the reference signal. Design virtual control quantity for:

[0055]

[0056] in, represents the adaptive parameter estimates associated with the optimal weights of the neural network, represents the neural network basis function vector, k1>0, λ1>0 is a designed normal number.

[0057] (b) Define the tracking error as Design virtual control volume for:

[0058]

[0059] in, represents the adaptive parameter estimates associated with the optimal weights of the neural network, represents the neural network basis function vector, k i >0,λ i >0 is a designed normal number.

[0060] (c) Define the tracking error as The actual control quantity u is designed as:

[0061]

[0062] in, represents the adaptive parameter estimates associated with the optimal weights of the neural network, represents the neural network basis function vector, k n >0,λ n >0 is a designed normal number.

[0063] Design parameters The adaptive law is:

[0064]

[0065] Among them, σ>0 and η>0 are designed normal constants;

[0066] Step 3: Design of auxiliary controller based on ADP;

[0067] The controller u designed in step 2 is returned to the system (1) to obtain the output prediction value y pre , define the prediction tracking error as e pre =y pre -y r .

[0068] The design evaluation network is:

[0069]

[0070] Where z(t)=[e pre (t-1),e pre (t)] T represents the forecast tracking error, u a (t)=[u a1 (t),…,u am (t)] T represents the auxiliary control input, p cj (t) and q cj (t) represents the input and output of the jth hidden layer node of the evaluation network, represents the weight from the i-th input layer to the j-th hidden layer of the evaluation network, Represents the weight between the hidden and output layers of the evaluation network, is the estimated value of the cost function J(t), Where 0<β≤1 represents the discount factor, r(t)=z(t) T Qz(t)+u a (t) T Ru a (t) represents the utility function, Q∈R 2×2 , R∈R m×m is the designed positive definite symmetric matrix.

[0071] The design execution network is:

[0072]

[0073] Where: p aj (t) and q aj (t) represents the input and output of the jth hidden layer node of the execution network, u ak (t) represents the kth control variable output by the execution network, represents the weight of executing the network's ith input to the jth hidden layer, Represents the weights from the kth hidden layer to the jth output layer of the execution network.

[0074] The weight update law of the design evaluation network is:

[0075] w c (t+1)=w c (t)-μ c βe c (t)φ cj (t) (17)

[0076] Where: μ c >0 is the designed evaluation network learning rate.

[0077] The weight update law of the designed execution network is:

[0078]

[0079] Among them: The element in row i and column k of matrix C(t) is μ a >0 is the designed execution network learning rate.

[0080] Step 4: Integrate the basic controller and the auxiliary controller to control the system.

[0081] According to the control input u obtained in step 2 and the control input u obtained in step 3 a The total control input U = u + u a , the control input U is returned to the flight dynamics model (1) to implement system command tracking control.

[0082] Example:

[0083] The present invention is further described in conjunction with a calculation example of an aircraft system.

[0084] Step 1: Model building;

[0085] The longitudinal dynamics model of the aircraft considering unknown time-varying disturbances is:

[0086]

[0087] Where V, γ, α and q represent velocity, track angle, angle of attack and pitch rate respectively; T, L and M yyRespectively represent engine thrust, lift and pitch moment; m, I yy and g represent the mass of the aircraft, the moment of inertia of the y-axis, and the acceleration due to gravity, respectively; δ e is the elevator deflection angle; d γ =0.001sin(0.2t),d α =0.003cos(0.1t),d q = 0.001sin(0.3t) represents the disturbance in the three subsystems respectively; the calculation method of the relevant aerodynamic force and moment in (10) is as follows

[0088]

[0089] Among them, S w represents the wing area; Q represents the dynamic pressure; ρ h Indicates air density; c A represents the average aerodynamic chord length; C L and C m They represent lift coefficient and pitch moment coefficient respectively; C D0 , C Dα , C L0 , C Lα , C m0 , C mα , and C mq denotes the corresponding aerodynamic derivative.

[0090] Define x1 = γ, x2 = α, x3 = q, then the system can be described as follows in strict feedback form

[0091]

[0092] in, u=δ e is the system control input, and Indicates about An unknown smooth function of

[0093] Step 2: Basic controller design based on backstepping method;

[0094] (a) According to formula (1), the track angle tracking error is defined as e1 = yy r , where y r Represents the reference signal. Design virtual control quantity for

[0095]

[0096] in, represents the adaptive parameter estimates associated with the optimal weights of the neural network, Represents the neural network basis function vector, and the design parameters are k1=10, λ1=50.

[0097] (b) Define the angle of attack tracking error as Design virtual control volume for:

[0098]

[0099] in, represents the adaptive parameter estimates associated with the optimal weights of the neural network, Represents the neural network basis function vector, and the design parameter values ​​are k2>10, λ2>50.

[0100] (c) Define the pitch angular velocity tracking error as The actual control quantity u is designed as:

[0101]

[0102] in, represents the adaptive parameter estimates associated with the optimal weights of the neural network, Represents the neural network basis function vector, and the design parameters are k3=20, λ3=50.

[0103] Design parameters The adaptive law is:

[0104]

[0105] Among them, σ>0 and η>0 are designed normal constants.

[0106] Step 3: Design of auxiliary controller based on ADP;

[0107] The controller u designed in step 2 is returned to the system (1) to obtain the output prediction value y pre , define the prediction tracking error as e pre =y pre -y r .

[0108] The design evaluation network is:

[0109]

[0110] Where z(t)=[e pre (t-1),e pre (t)] T represents the forecast tracking error, u a (t)=[ua1 (t),…,u am (t)] T represents the auxiliary control input, p cj (t) and q cj (t) represents the input and output of the jth hidden layer node of the evaluation network, represents the weight from the i-th input layer to the j-th hidden layer of the evaluation network, Represents the weight between the hidden and output layers of the evaluation network, is the estimated value of the cost function J(t), The discount factor is β = 0.9, r(t) = z(t) T Qz(t)+u a (t) T Ru a (t) represents the utility function, Q∈R 2×2 , R∈R m×m is the designed positive definite symmetric matrix.

[0111] The design execution network is:

[0112]

[0113] Where: p aj (t) and q aj (t) represents the input and output of the jth hidden layer node of the execution network, u ak (t) represents the kth control variable output by the execution network, represents the weight of executing the network's ith input to the jth hidden layer, Represents the weights from the kth hidden layer to the jth output layer of the execution network.

[0114] The weight update law of the design evaluation network is:

[0115] w c (t+1)=w c (t)-μ c βe c (t)φ cj (t) (28)

[0116] Among them: The design value of the evaluation network learning rate is μ c =2.

[0117] The weight update law of the designed execution network is:

[0118]

[0119] Among them: The element in row i and column k of matrix C(t) is The design value of the execution network learning rate is μ a =2 Step 4: Integrate the basic controller and the auxiliary controller to control the system.

[0120] According to the control input u obtained in step 2 and the control input u obtained in step 3 a The total control input U = u + u a , the control input U is returned to the flight dynamics model equation (12) to achieve system command tracking control.

Claims

1. A self-learning optimization control method for an aircraft based on adaptive dynamic programming, characterized in that: The steps include: Step 1: Model building; Consider the following strict feedback system: in, i=1,…,n, u∈R represents the system input, y∈R represents the system output, Indicates about An unknown smooth function of represents a known non-zero function, d i (t) represents the time-varying disturbance, satisfying in and Represents the upper bound of disturbance and disturbance change rate; Step 2: Basic controller design based on backstepping method; Step 2-1: According to formula (1), define the tracking error as e1 = yy r , where y r Represents reference signal; designs virtual control quantity for: in, represents the adaptive parameter estimates associated with the optimal weights of the neural network, represents the neural network basis function vector, k1>0, λ1>0 is a designed normal constant; Step 2-2: Define the tracking error as Design virtual control volume for: in, Represents the neural network basis function vector; Step 2-3: Define the tracking error as The actual control quantity u is designed as: in, Represents the neural network basis function vector; Design parameters The adaptive law is: Among them, σ>0 and η>0 are designed normal constants; Step 3: Design of auxiliary controller based on ADP; Step 3-1: Return the controller u designed in step 2 to system (1) to obtain the output prediction value y pre , define the prediction tracking error as e pre =y pre -y r ; Step 3-2: Design the evaluation network as follows: Where z(t)=[e pre (t-1),e pre (t)] T represents the forecast tracking error, u a (t)=[u a1 (t),…,u am (t)] T represents the auxiliary control input, p cj (t) and q cj (t) represents the input and output of the jth hidden layer node of the evaluation network, represents the weight from the i-th input layer to the j-th hidden layer of the evaluation network, Represents the weight between the hidden and output layers of the evaluation network, is the estimated value of the cost function J(t), Where 0<β≤1 represents the discount factor, r(t)=z(t) T Qz(t)+u a (t) T Ru a ( t ) represents the utility function, Q∈R 2×2 , R∈R m×m is the designed positive definite symmetric matrix; Step 3-3: Design the execution network as follows: Where: p aj (t) and q aj (t) represents the input and output of the jth hidden layer node of the execution network, u ak (t) represents the kth control variable output by the execution network, represents the weight of executing the network's ith input to the jth hidden layer, Represents the weights from the kth hidden layer to the jth output layer of the execution network; Step 3-4: Design the weight update law of the evaluation network as: w c (t+1)=w c (t)-μ c βe c (t)φ cj (t) (8) Where: μ c >0 is the designed evaluation network learning rate; Step 3-5: Design and execute the network weight update law as follows: Among them: The element in row i and column k of matrix C(t) is μ a >0 is the designed execution network learning rate; Step 4: Integrate the basic controller and the auxiliary controller to control the system; According to the control input u obtained in step 2 and the control input u obtained in step 3 a The total control input U = u + u a , the control input U is returned to the flight dynamics model equation (1) to achieve system command tracking control.

2. The method for self-learning optimization control of an aircraft based on adaptive dynamic programming according to claim 1, characterized in that: In the aircraft self-learning optimization control method based on adaptive dynamic programming, the aircraft longitudinal dynamics model containing unknown time-varying interference is constructed as follows: Where V, γ, α and q represent velocity, track angle, angle of attack and pitch rate respectively; T, L and M yy Respectively represent engine thrust, lift and pitch moment; m, I yy and g represent the mass of the aircraft, the moment of inertia of the y-axis, and the acceleration due to gravity, respectively; δ e is the elevator deflection angle; d γ =0.001sin(0.2t),d α =0.003cos(0.1t),d q =0.001sin(0.3t) represents the interference in the three subsystems respectively.

Images