An intelligent control method for hypersonic deformable aircraft with weak model dependence

Abstract

The present invention belongs to the field of aerospace vehicle control, and relates to an intelligent control method for a hypersonic deformable aircraft with weak model dependence. The present invention first establishes a control-oriented hypersonic deformable aircraft dynamics model for the preparation of intelligent control. Afterwards, an integral reinforcement learning algorithm with weak model dependence is designed to construct a basic framework of intelligent control and reduce the degree of dependence on the model of the deformable aircraft. Furthermore, an adaptive learning law for policy gradient is designed, and the key parts of the control law are updated through policy gradient. Combined with the update idea of ​​the data stack, the data pressure of online updates is reduced and the update speed of the control system is improved. This method is an intelligent control method that reduces the degree of dependence of the control method on the model, and has broad application prospects.

Description

Technical Field

[0001] The invention belongs to the field of aerospace vehicle control and relates to an intelligent control method for a hypersonic deformable aircraft with weak model dependence. Background Art

[0002] The characteristics of hypersonic morphing vehicles include fast flight, long range, and strong penetration capability. By optimizing the aerodynamic configuration, this vehicle not only improves the aerodynamic characteristics, but also expands the flight envelope, improves energy efficiency, and enhances environmental adaptability and flexibility in mission adjustment. Although the variability capability enhances the aerodynamic characteristics, it also adds challenges to the design of the control system, especially during the deformation process, the strong coupling, nonlinearity, and multimodal characteristics of the vehicle make precise control more difficult. Therefore, it is crucial to develop a robust controller, reduce the frequency of data transmission and actuators, and ensure the stability of the system attitude.

[0003] Considering the high nonlinearity, strong time-varying characteristics, strong coupling characteristics and large uncertainty of hypersonic deformable aircraft, and the strong adjustment ability to complex flight environment and flight mission adjustment, traditional control methods are difficult to meet the high stability, robustness and control accuracy requirements. Flight control in the deformation stage will cause pressure on the data transmission of the aircraft system and frequent actions of the actuators, further increasing the difficulty of control system design. Control methods based on advanced control theory all rely on high-precision modeling of the controlled object, which is difficult to meet the attitude control needs of modern high-speed aircraft. Therefore, in order to fundamentally solve the contradiction between the existing attitude control methods and actual needs, it is necessary to reduce the dependence of the control method on the model, so as to improve the adaptability to model uncertainty and enhance the versatility to different aerodynamic shapes. It is necessary to establish a set of weak model-dependent intelligent control methods for hypersonic deformable aircraft.

[0004] Song Shenmin, Sun Jingguang, Wang Yan, etc. introduced a new continuously differentiable nonlinear saturation function in "A hypersonic aircraft tracking control method based on nonlinear functions" (CN108427289B), and combined with adaptive theory, designed a nonlinear robust controller to solve the problem of complex control models and poor robustness of existing aircraft. However, this patent is based on adaptive control theory for real-time control, and its research object does not consider the deformation effect of hypersonic aircraft, and the method involved needs to cover continuously differentiable nonlinear saturation functions, and does not consider the flight control problem when the model information is less, and the focus on control quality is relatively weak.

[0005] Hu Qinglei, Li Ziming, Guo Lei, etc. constructed a smooth function to estimate the nonlinear input saturation in the "Neural Network-Based Hypersonic Aircraft Adaptive Compensation Control Method" (CN108375907A), and introduced a radial basis function neural network to estimate the nonlinear function in the longitudinal dynamics model of the hypersonic aircraft. The adaptive compensation controller and the corresponding adaptive parameter update law of the hypersonic aircraft were designed by backstepping, which solved the impact of various elevator failures and actuator saturation on the aircraft during the flight of the hypersonic aircraft, and ensured the fault tolerance and robustness of the system. This patent uses a radial basis function neural network to estimate the nonlinear function in the longitudinal dynamics model of the hypersonic aircraft. The method does not explain how to correct and compensate for the deformation of the hypersonic aircraft. The design of the update law leads to frequent online data calculations, which is not suitable for strong adaptive control under high dynamics. In addition, this method requires the model to have an affine form to a certain extent, and the control effect is less considered, which has certain limitations.

[0006] In "Output Error Limited and Input Saturation Control Method for Morphing Aircraft" (see "Acta Aeronautica Sinica", August 2023, No. 15, pp. 400-411), Chen Haolan, Wang Peng, and Tang Guojian obtained a control system for input saturation based on the hyperbolic tangent function and auxiliary system, and designed a conversion error so that the system can achieve output error limited control under more relaxed convergence conditions. This paper uses the conversion error to enable the system to achieve output error limited control, but this method is not suitable for the control requirements of hypersonic morphing aircraft with non-affine characteristics. The dynamic nonlinearity depends not only on the state of the system, but also on the control input, and cannot adapt to the use of deformation control tasks.

[0007] In "Research on Robust Control Methods for Elastic Hypersonic Deformable Vehicles" (see "National University of Defense Technology", 2021 Master's Thesis, pages 38-50), He Sheng proposed an adaptive control method for elastic hypersonic deformable vehicles based on neural networks. The neural network is used to perform nonlinear approximation on the unknown model, and the overall controller design is completed in combination with the idea of ​​dynamic surface backstepping control, which can achieve stable control of the deformation process. However, this method is not suitable for the control requirements of hypersonic deformable vehicles with non-affine characteristics, and the introduction of backstepping control will increase the amount of algorithm calculations, and the large number of control parameters makes the actual use of the controller more complicated.

[0008] Wang Zijian, Zhang Shuyu, and Hou Mingzhe designed an online time-varying parameter identification algorithm in "Variant Aircraft Control Based on Online Parameter Identification" (see "Journal of Ordnance Equipment Engineering", October 2022, Issue 10, Pages 60-65) to determine the impact of factors such as the uncertainty of aerodynamic coefficients caused by shape changes and external disturbances on the aircraft, and ensure the maneuverability of the aircraft through a state feedback control strategy with fixed time convergence. However, this method relies on the accuracy of system modeling, which may lead to a contradiction between the attitude control method and the actual flight mission requirements, making the control accuracy limited by the identification accuracy and modeling accuracy. There is a certain degree of difficulty in applying this method when dealing with control problems with strong time-varying characteristics, strong coupling characteristics, and high uncertainty. Summary of the invention

[0009] The weak model-dependent intelligent control method proposed in this invention for designing a hypersonic deformable aircraft fully considers the high nonlinearity, strong time-varying characteristics, strong coupling characteristics and large uncertainty of the hypersonic deformable aircraft. By designing a weak model-dependent intelligent control method, the control method reduces the degree of dependence on the model, so as to improve the adaptability to model uncertainty and enhance the versatility of different aerodynamic shapes, fundamentally solving the contradiction between the existing attitude control method and actual needs. By designing a two-step update strategy based on data stack, the pressure on aircraft system data transmission caused by flight control in the deformation stage and the frequent action of the actuator are dealt with, and the operability of the online application of the control system is further enhanced.

[0010] The technical solution of the present invention is as follows:

[0011] A method for intelligent control of a hypersonic deformable aircraft with weak model dependence comprises the following steps:

[0012] (1) Establishment of a control-oriented hypersonic deformable vehicle dynamics model

[0013] In order to adapt to the mission requirements of different working conditions, the longitudinal model of the variable-sweep aircraft is expressed as follows:

[0014]

[0015] Where: V is the speed, is the derivative of velocity, h is the height, is the derivative of altitude, α is the angle of attack, is the derivative of the angle of attack, θ is the pitch angle, is the derivative of the pitch angle, q is the pitch rate, is the derivative of the pitch rate, I yy is the pitch moment of inertia, m is the mass of the aircraft; T is the engine thrust, D is the flight resistance, L is the aircraft lift, and M is the pitch moment, and is expressed as:

[0016]

[0017] Where: is the thrust coefficient, δ t is the fuel equivalence ratio, δ e is the elevator command. ρ is the air density, S w is the wing reference area, c A is the average geometric chord length, C L (α,δ e ,ξ b ,ξ s ) represents the lift coefficient, C D (α,ξ b ,ξ s ) represents the drag coefficient, C M (α,δ e ,ξ b ,ξ s ) represents the pitching moment coefficient.

[0018] ξ b is the span deformation rate, and its value range is ξ b ∈[0,1],ξ s is the sweep angle deformation rate, and its value range is ξ s ∈[0,1], respectively defined as follows:

[0019]

[0020] Where: b is the wingspan; b min and b max are the wingspan of the aircraft without deformation and the wingspan of the aircraft with maximum deformation, respectively; s is the sweep angle; s min and max They are the sweep angles when the aircraft is not deformed and when it is at maximum deformation. L , drag coefficient C D , pitch moment coefficient C M The functional relationship is defined as follows:

[0021]

[0022] Where: is the coefficient relationship between lift and deformation when the angle of attack is 0, is the coefficient relationship between lift and angle of attack, is the coefficient relationship between lift and elevator command; is the coefficient relationship between resistance and deformation when the angle of attack is 0, is the coefficient relationship between drag and angle of attack, is the coefficient relationship between the drag and the square of the angle of attack; is the coefficient relationship between the pitch moment and the deformation when the angle of attack is 0, is the coefficient relationship between pitch moment and angle of attack, is the coefficient relationship between the pitch moment and the elevator command.

[0023] (2) Designing an integral reinforcement learning algorithm with weak model dependence

[0024] The wingspan deformation rate ξ b and the sweep angle deformation rate ξ s As an additional control input, the following system model is obtained:

[0025]

[0026] Where: x(t) = [V, α, θ, q, h] T is the state vector, is the derivative of the state vector; u(t) = [δ e ,δ t ,ξ b ,ξ s ] T is the control input vector; f(x(t),u(t)) describes the functional relationship between the first-order derivative of the state vector and the state and input, y(t)=h(x)=[V,α,θ,q,h] T is the output vector.

[0027] The system model of formula (5) is a non-affine system, so the integrated reinforcement learning strategy of the hypersonic deformable aircraft based on the actor-critic network is designed as follows:

[0028] First, the expression of the Q function is given:

[0029]

[0030] Among them, Q(x,u) represents the function related to the state quantity x(t) and the control input quantity u(t), is the weight vector of the critic network, Indicates L c dimensional vector group, ω c,1 ,ω c,2 ,…, They represent the weight vector L c sub-vectors, are the basis vectors of the critic network, φ1(x,u), φ2(x,u),…, They represent the critic basis function L c sub-vectors.

[0031] Similarly, for the policy function there exists a set of weight vectors {ψ i(x)}, so that

[0032]

[0033] Among them, u(x) represents the control amount generated under the policy function, is the weight matrix of the actor network, Indicates L a A matrix with 4 rows and columns, w a,1 ,w a,2 ,…, L represents the actor network weight vector a sub-vectors; are the basis vectors of the actor network, Indicates L a A vector with 1 row and 1 column, ψ1(x), ψ2(x),…, They represent the L of the actor network basis vectors respectively. a sub-vectors. Define the critic network approximation error as ε c (x,u), define the actor network approximation error as ε a (x), these two errors approach 0 as the amount of information contained in the vector matrix increases.

[0034] Then, define the optimal action-value function Q * and the optimal action value u * Since the ideal weights are unknown, the corresponding Q * and u * It is also difficult to calculate. Therefore, the critic network and actor network are used for approximation, which can be expressed as follows:

[0035]

[0036] in, represents the estimated value of the action-value function associated with the state quantity x and the control quantity u, represents the estimated value of the action associated with the state quantity x, and They are the critic network weights W c and the actor network weights W a Based on the actor-critic structure, the control quantity u of the policy iterative integral reinforcement learning in formula (8) is optimized and adjusted.

[0037] (3) Designing an adaptive learning law for policy gradients

[0038] The policy gradient-based integral reinforcement learning algorithm can be implemented by the reinforcement learning algorithm of function approximation in the actor and critic network. The policy evaluation is expressed as

[0039]

[0040] Where: T s Represents the time interval of each iteration. x(tT s ) and u(t- T s ) represents the state and control quantity of the previous moment, Φ(x(t- T s ),u(t- T s )) represents the basis function of the previous moment, Indicates T s The critic network strategy evaluation effect within the time interval. r(x(τ),u(τ)) represents a continuously differentiable and positive definite real-valued function. The action-value function in formula (8) can be updated by recursive least squares or batch least squares Parameters.

[0041] Based on the idea of ​​policy iteration, the gradient descent method is used to update the actor network parameters:

[0042]

[0043] In the formula, σ is the positive parameter to be designed, represents the estimated value of the actor neural network at the current i-th moment, represents the critic neural network weight at the current moment, Ψ(x) is the basis vector of the actor network, Represents the gradient information of the actor neural network about the control amount u at the current i-th moment.

[0044] In particular, the iteration time interval T s The value of can be changed depending on how long it takes to receive meaningful information from the observations. To this end, define a three-dimensional storage stack of actor network data It has L a Rows, 4 columns, and maximum number of pages Define a 2D stack of critic data It has (L a +2L c ) rows, and the maximum number of columns Among them, for the current sequence τ, we have and

[0045] Design a two-step update strategy based on the data stack:

[0046]

[0047] Where: The latest stack of the ιth column is defined as and in and * , Φ * Represents the latest values ​​of the actor network data storage stack and the critic data stack. Represents the minimum singular value, whose size can be calculated by singular value decomposition. Define the current time as t k , the corresponding next moment is t k+1 Since the maximum dimension of the online data stack is The values ​​in each dimension are different. The time corresponding to the maximum value stack, using the target column of the latest flight data m right Update and meet Indicates the maximum value within the obtained value range. a is a positive constant to be designed.

[0048] For the control quantity obtained by formula (8), in order to facilitate the stability proof, the control strategy u generated at the i-th moment is defined as i (x), there exists a real-valued function r(x,u i (x)) satisfies continuous differentiability and positive definiteness, so the value function Satisfies the following relationship

[0049]

[0050] Among them, f(x,u i (x)) is the dynamic relationship corresponding to the i-th moment. There exists a positive constant σ such that

[0051] Selecting the value function is the Lyapunov function. Based on formula (12), The derivative is

[0052]

[0053] Therefore, the equilibrium point of the closed-loop system is asymptotically stable, which means that the control strategy u (i+1) It is feasible.

[0054] Beneficial effects of the present invention:

[0055] The technical advantages of the present invention are:

[0056] 1. Taking into account the high nonlinearity, strong time-varying characteristics, strong coupling characteristics and large uncertainty of hypersonic deformable aircraft, a weak model-dependent intelligent control method is designed to reduce the control method's dependence on the model, so as to improve the adaptability to model uncertainty and enhance the versatility to different aerodynamic shapes, fundamentally resolving the contradiction between the existing attitude control method and the actual flight mission requirements.

[0057] 2. By designing a two-step update strategy based on data stack, we can address the problems of heavy data transmission pressure on the aircraft system and frequent actions of actuators caused by flight control in the deformation phase, further enhancing the operability of the control system's online application.

[0058] 3. This method is applicable to the control requirements of hypersonic deformable vehicles with non-affine characteristics, whose dynamic nonlinearity depends not only on the state of the system but also on the control input, which is crucial to the realization of the deformation control task.

[0059] The key points of this application are:

[0060] 1. A control method based on weak model dependence can achieve adaptability to the uncertainty of the aircraft model and enhance the versatility for different aerodynamic shapes.

[0061] 2. A two-step update strategy based on the data stack is used to address the problems of data transmission pressure on the aircraft system caused by flight control during the deformation phase, as well as frequent actions of actuators, and to alleviate the computational burden brought by frequent online adaptive updates.

[0062] 3. Integral reinforcement learning is used as the intelligent control method, which comprehensively considers multiple factors such as flight quality, control performance and online optimization. It also has the characteristics of simple design and easy implementation. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is an integrated reinforcement learning strategy for hypersonic morphing vehicles based on an actor-critic network;

[0064] Figure 2 It is a flow chart for the implementation of the intelligent control scheme for a hypersonic deformable aircraft with weak model dependence;

[0065] Figure 3 It is a simulation curve diagram of the speed tracking of a hypersonic deformable aircraft;

[0066] Figure 4 is a time graph of the velocity tracking error of a hypersonic deformable vehicle;

[0067] Figure 5 It is a curve diagram of the simulation of the height tracking of the hypersonic deformable aircraft;

[0068] Figure 6is a time graph of the height tracking error of a hypersonic deformable vehicle;

[0069] Figure 7 It is a graph of the fuel equivalence ratio of a hypersonic shape-shifting vehicle over time;

[0070] Figure 8 It is a time curve diagram of elevator command of hypersonic shape-shifting aircraft;

[0071] Fig. 9 is a time curve of the wingspan deformation rate of a hypersonic deformable aircraft;

[0072] Fig.10 It is a time curve of the sweep angle deformation rate of a hypersonic deformable vehicle. DETAILED DESCRIPTION

[0073] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.

[0074] like Figure 2 As shown, the invention provides a method for intelligently controlling a hypersonic deformable aircraft with weak model dependence, and its implementation steps are as follows:

[0075] Step 1: Establish a control-oriented hypersonic shape-shifting vehicle dynamics model for preparation of intelligent control.

[0076] Step 2: Design an integral reinforcement learning algorithm with weak model dependence, build a basic framework for intelligent control, and reduce the model dependence of the deformable aircraft.

[0077] Step 3: Design an adaptive learning law for policy gradients, update the key parts of the control law through policy gradients, combine the update ideas of the data stack, reduce the data pressure of online updates, and improve the update speed of the control system.

[0078] Step 4: Evaluate the performance of the intelligent control method, combine it with simulation software, adjust parameters within the allowable range, and complete simulation calculations and performance tests.

[0079] The specific steps are as follows:

[0080] Step 1: Establish a control-oriented hypersonic shape-shifting vehicle dynamics model

[0081] In order to adapt to the mission requirements of different working conditions, the longitudinal model of the variable-sweep aircraft is expressed as follows:

[0082]

[0083] Where: V is the speed, is the derivative of velocity, h is the height, is the derivative of altitude, α is the angle of attack, is the derivative of the angle of attack, θ is the pitch angle, is the derivative of the pitch angle, q is the pitch rate, is the derivative of the pitch rate, I yy is the pitch moment of inertia, m is the mass of the aircraft; T is the engine thrust, D is the flight resistance, L is the aircraft lift, and M is the pitch moment, and is expressed as:

[0084]

[0085] Where: is the thrust coefficient, δ t is the fuel equivalence ratio, δ e is the elevator command. ρ is the air density, S ω is the wing reference area, c A is the average geometric chord length, C L (α,δ e ,ξ b ,ξ s ) represents the lift coefficient, C D (α,ξ b ,ξ s ) represents the drag coefficient, C M (α,δ e ,ξ b ,ξ s ) represents the pitching moment coefficient.

[0086] ξ b is the span deformation rate, and its value range is ξ b ∈[0,1],ξ s is the sweep angle deformation rate, and its value range is ξ s ∈[0,1], respectively defined as follows:

[0087]

[0088] Where: b is the wingspan; b min and b max are the wingspan of the aircraft without deformation and the wingspan of the aircraft with maximum deformation, respectively; s is the sweep angle; s min and max They are the sweep angles when the aircraft is not deformed and when it is at maximum deformation. L , drag coefficient C D , pitch moment coefficient C M The functional relationship is defined as follows:

[0089]

[0090] Where: is the coefficient relationship between lift and deformation when the angle of attack is 0, is the coefficient relationship between lift and angle of attack, is the coefficient relationship between lift and rudder down command; is the coefficient relationship between lift and deformation when the angle of attack is 0, is the coefficient relationship between lift and angle of attack, is the coefficient relationship between lift and the square of angle of attack; is the coefficient relationship between the pitch moment and the deformation when the angle of attack is 0, is the coefficient relationship between pitch moment and angle of attack, is the coefficient relationship between lift and rudder down command.

[0091] Step 2: Design an integral reinforcement learning algorithm with weak model dependence

[0092] The wingspan deformation rate ξ b , sweep angle deformation rate ξ s As an additional control input, the following model is obtained:

[0093]

[0094] Where: x(t) = [V, α, θ, q, h] T is the state vector, is the derivative of the state vector; u(t) = [δ e ,δ t ,ξ b ,ξ s ] T is the input vector; f(x(t),u(t)) describes the functional relationship between the first-order derivative of the state vector and the state and input, y(t)=h(x)=[V,α,θ,q,h] T is the output vector.

[0095] The above system is a non-affine system, so an integrated reinforcement learning strategy for a hypersonic deformable aircraft based on an actor-critic network can be designed, such as Figure 1 shown.

[0096] First, the expression of the Q function is given:

[0097]

[0098] Among them, Q(x,u) represents the function related to the state quantity x and the control quantity u, Indicates L c dimensional vector group, ω c,1 ,ω c,2 ,…, They represent the weight vector L c sub-vectors, in this embodiment, L is selected c =50. Similarly, are the basis vectors of the critic network, φ1(x,u), φ2(x,u),…, They represent the 50 sub-vectors of the critic basis function.

[0099] Similarly, for the policy function there exists a set of weight vectors {ψ i (x)}, so that

[0100]

[0101] Among them, u(x) represents the control amount generated under the policy function, is the weight matrix of the actor network, Indicates L a A matrix with 4 rows and columns, w a,1 ,w a,2 ,…, They represent the actor network weight vector L a sub-vectors, in this embodiment, L is selected a =200. Similarly, are the basis vectors of the actor network, Indicates L a A vector with 1 row and 1 column, ψ1(x), ψ2(x),…, They represent the 200 sub-vectors of the actor network basis vector. Define the critic network approximation error as ε c (x,u), define the actor network approximation error as ε a (x), these two errors approach 0 as the amount of information contained in the vector matrix increases.

[0102] Define the optimal action-value function Q * and the optimal action value u * Since the ideal weights are unknown, the corresponding Q * and u * It is also difficult to calculate. Therefore, the critic network and actor network are used for approximation, which can be expressed as follows:

[0103]

[0104] in, represents the estimated value of the action-value function associated with the state quantity x and the control quantity u, represents the estimated value of the action associated with the state quantity x, and They are the critic network weights W c and the actor network weights W a Based on the actor-critic structure, the controller of the policy iterative integral reinforcement learning in formula (21) is optimized and tuned.

[0105] Step 3: Design an adaptive learning law for policy gradient.

[0106] The policy gradient-based integral reinforcement learning algorithm can be implemented online through the reinforcement learning technique of function approximation in the actor and critic network. The policy evaluation is expressed as:

[0107]

[0108] Where: T s Indicates the time interval of each iteration. In this embodiment, T is selected s =0.05s. x(tT s ) and u(tT s ) represents the state and control quantity of the previous moment, Φ(x(tT s ),u(tT s )) represents the basis function of the previous moment, represents the critic network strategy evaluation effect within a 0.05s time interval. r(x(τ),u(τ)) represents a continuously differentiable and positive definite real-valued function. The action value function in formula (8) can be updated by recursive least squares or batch least squares Parameters.

[0109] Based on the idea of ​​policy iteration, the gradient descent method is used to update the actor network parameters:

[0110]

[0111] In the formula, σ is the positive parameter to be designed, which is selected as 0.1. represents the estimated value of the actor neural network at the current i-th moment, represents the critic neural network weight at the current moment, Ψ(x) is the basis vector of the actor network, Represents the gradient information of the actor neural network about the control amount u at the current i-th moment.

[0112] Define a three-dimensional storage stack for actor network data It has L a = 200 rows, 4 columns, and maximum number of pages Define a 2D stack of critic data It has (L a +2L c ) = 300 rows and the maximum number of columns Among them, for the current sequence τ, we have and

[0113] Design a two-step update strategy based on the data stack:

[0114]

[0115] Where: The latest stack of the ιth column is defined as and in and * , Φ * Represents the latest values ​​of the actor network data storage stack and the critic data stack. Represents the minimum singular value, whose size can be calculated by singular value decomposition. Define the current time as t k , the corresponding next moment is t k+1 Since the maximum dimension of the online data stack is The values ​​in each dimension are different. The time corresponding to the maximum value stack, using the target column of the latest flight data m right Update and meet Indicates the maximum value within the obtained value range. a is a positive constant to be designed.

[0116] For the control quantity obtained by formula (8), in order to facilitate the stability proof, the control strategy u generated at the i-th moment is defined as i (x), there exists a real-valued function r(x,u i (x)) satisfies continuous differentiability and positive definiteness, so the value function Satisfies the following relationship

[0117]

[0118] Among them, f(x,u i (x)) is the dynamic relationship corresponding to the i-th moment. There exists a positive constant σ such that

[0119] Selecting the value function is the Lyapunov function, and its derivative gives

[0120]

[0121] in, The equilibrium point of the closed-loop system is asymptotically stable, which means that the control strategy u (i+1) It is feasible.

[0122] Step 4: Verify the performance of the intelligent control method

[0123] A designed weak model-dependent intelligent control method is applied to a hypersonic deformable aircraft system with the help of commonly used computer numerical calculation and simulation software. After selecting the design parameters within the allowable range, simulation calculations are performed and performance tests are carried out. If the performance of the weak model-dependent intelligent control method meets the requirements, the design is completed; otherwise, the design parameters need to be adjusted, and simulation calculations are performed again and performance tests are carried out.

[0124] In order to verify the feasibility of the proposed weak model-dependent hypersonic deformable aircraft intelligent control method, the following simulation scenario design was carried out. The reference command consists of three basic maneuvering processes: in the first 200s, the speed and altitude reference commands are changed from the initial state to the expected value 1; in the next 200s, the speed and altitude reference commands are changed from the expected value 1 to the expected value 2; in the last 400s, the speed and altitude reference commands are changed from the expected value 2 to the expected value 3. The initial state of the aircraft and the expected values ​​at different stages in the simulation test are given in Table 1. The initial values ​​of the control quantities are designed as follows: the initial value of the fuel equivalence ratio is 0, the initial value of the elevator command is 0, the initial value of the wingspan deformation rate is 0.48, and the initial value of the sweep angle deformation rate is 0.5.

[0125] The reference signal is transmitted to the control system through a second-order command filter. The damping ratio of the second-order command filter used is 0.75 and the natural frequency is 0.04 rad / s.

[0126] Table 1 Initial value and expected value of flight status

[0127]

[0128] The result is as follows Figure 3-Figure 10 As shown in Figure 2, it can be seen that under the proposed weak model-dependent control scheme, the speed and altitude of the hypersonic deformable aircraft are effectively tracked, as shown in Figure 2. Figure 3 and Figure 5 In addition, Figure 4 and Figure 6 The corresponding tracking errors are given, which more clearly demonstrates that the control scheme can achieve stable tracking with high precision. Figure 7-10 The time variation curves of four control commands are given, including fuel equivalence ratio, elevator command, wingspan deformation rate, and sweep angle deformation rate. Among them, the fuel equivalence ratio plays an important role in speed tracking control, and the control command is within a reasonable range. The elevator command, wingspan deformation rate, and sweep angle deformation rate are stable in altitude tracking control, giving full play to the deformation ability of the deformable aircraft. More importantly, the designed control scheme fully considers the strong coupling, strong nonlinearity and non-affine characteristics of the hypersonic deformable aircraft, adopts an integrated intelligent control scheme, takes into account multiple actuators and gives full play to the performance, and achieves good tracking control of the task.

Claims

1. A method for intelligent control of a hypersonic deformable aircraft with weak model dependence, characterized in that: The steps include: (1) Establishment of a control-oriented hypersonic deformable vehicle dynamics model In order to adapt to the mission requirements of different working conditions, the longitudinal model of the variable-sweep aircraft is expressed as follows: Where: V is the speed, is the derivative of velocity, h is the height, is the derivative of altitude, α is the angle of attack, is the derivative of the angle of attack, θ is the pitch angle, is the derivative of the pitch angle, q is the pitch rate, is the derivative of the pitch rate, I yy is the pitch moment of inertia, m is the mass of the aircraft; T is the engine thrust, D is the flight resistance, L is the aircraft lift, and M is the pitch moment, and is expressed as: Where: is the thrust coefficient, δ t is the fuel equivalence ratio, δ e is the elevator command; ρ is the air density, S w is the wing reference area, c A is the average geometric chord length, C L (α,δ e ,ξ b ,ξ s ) represents the lift coefficient, C D (α,ξ b ,ξ s ) represents the drag coefficient, C M (α,δ e ,ξ b ,ξ s ) represents the pitching moment coefficient; ξ b is the span deformation rate, and its value range is ξ b ∈[0,1],ξ s is the sweep angle deformation rate, and its value range is ξ s ∈[0,1], respectively defined as follows: Where: b is the wingspan; b min and b max are the wingspan of the aircraft without deformation and the wingspan of the aircraft with maximum deformation, respectively; s is the sweep angle; s min and max are the sweep angles when the aircraft is not deformed and when it is at maximum deformation; the lift coefficient C L , drag coefficient C D , pitch moment coefficient C M The functional relationship is defined as follows: Where: is the coefficient relationship between lift and deformation when the angle of attack is 0, is the coefficient relationship between lift and angle of attack, is the coefficient relationship between lift and elevator command; is the coefficient relationship between resistance and deformation when the angle of attack is 0, is the coefficient relationship between drag and angle of attack, is the coefficient relationship between the drag and the square of the angle of attack; is the coefficient relationship between the pitch moment and the deformation when the angle of attack is 0, is the coefficient relationship between pitch moment and angle of attack, is the coefficient relationship between the pitch moment and the elevator command; (2) Designing an integral reinforcement learning algorithm with weak model dependence The wingspan deformation rate ξ b and the sweep angle deformation rate ξ s As an additional control input, the following system model is obtained: Where: x(t) = [V, α, θ, q, h] T is the state vector, is the derivative of the state vector; u(t) = [δ e ,δ t ,ξ b ,ξ s ] T is the control input vector; f(x(t),u(t)) describes the functional relationship between the first-order derivative of the state vector and the state and input, y(t)=h(x)=[V,α,θ,q,h] T is the output vector; The system model of formula (5) is a non-affine system, so the integrated reinforcement learning strategy of the hypersonic deformable aircraft based on the actor-critic network is designed as follows: First, the expression of the Q function is given: Among them, Q(x,u) represents the function related to the state quantity x(t) and the control input quantity u(t), is the weight vector of the critic network, Indicates L c dimensional vector group, They represent the weight vector L c sub-vectors, is the basis vector of the critic network, They represent the critic basis function L c sub-vectors; For the policy function there exists a weight vector set {ψ i (x)}, so that Among them, u(x) represents the control amount generated under the policy function, is the weight matrix of the actor network, Indicates L a A matrix with 4 rows and 4 columns, L represents the actor network weight vector a sub-vectors; are the basis vectors of the actor network, Indicates L a A vector with 1 row and 1 column, They represent the L of the actor network basis vectors respectively. a sub-vectors; define the critic network approximation error as ε c (x,u), define the actor network approximation error as ε a (x), these two errors approach 0 as the amount of information contained in the vector matrix increases; Then, define the optimal action-value function Q * and the optimal action value u * ; Since the ideal weights are unknown, the corresponding Q * and u * It is also difficult to calculate; therefore, the critic network and actor network are used for approximation, which can be expressed as follows: in, represents the estimated value of the action-value function associated with the state quantity x and the control quantity u, represents the estimated value of the action associated with the state quantity x, and They are the critic network weights W c and the actor network weights W a The estimated value of; Based on the actor-critic structure, the control quantity u of the policy iterative integral reinforcement learning in formula (8) is optimized and adjusted; (3) Designing an adaptive learning law for policy gradients The policy gradient-based integral reinforcement learning algorithm can be implemented by the reinforcement learning algorithm of function approximation in the actor and critic network; the policy evaluation is expressed as Where: T s represents the time interval of each iteration; x(tT s ) and u(tT s ) represents the state and control quantity of the previous moment, Φ(x(tT s ),u(tT s )) represents the basis function of the previous moment, Indicates T s The critic network strategy evaluation effect within the time interval; r(x(τ),u(τ)) represents a continuously differentiable and positive definite real-valued function; the action-value function in formula (8) can be updated by recursive least squares or batch least squares Parameters; Based on the idea of ​​policy iteration, the gradient descent method is used to update the actor network parameters: In the formula, σ is the positive parameter to be designed, represents the estimated value of the actor neural network at the current i-th moment, represents the critic neural network weight at the current moment, Ψ(x) is the basis vector of the actor network, Represents the gradient information of the actor neural network about the control amount u at the current i-th moment; Iteration time interval T s The value of changes depending on how long it takes to receive meaningful information from the observations; for this purpose, a three-dimensional storage stack of actor network data is defined It has L a Rows, 4 columns, and maximum number of pages Define a 2D stack of critic data It has (L a +2L c ) rows, and the maximum number of columns Among them, for the current sequence τ, we have and Design a two-step update strategy based on the data stack: Where: The latest stack of the ιth column is defined as and in and * , Φ * Represents the latest values ​​of the actor network data storage stack and the critic data stack; Represents the minimum singular value, whose size can be calculated by singular value decomposition; define the current time as t k , the corresponding next moment is t k+1 ; Since the maximum dimension of the online data stack is The values ​​in each dimension are different; definition The time corresponding to the maximum value stack, using the target column of the latest flight data m right Update and meet Represents the maximum value within the obtained value range; η a is a positive constant to be designed; For the control quantity obtained by formula (8), in order to facilitate the stability proof, the control strategy u generated at the i-th moment is defined as i (x), there exists a real-valued function r(x,u i (x)) satisfies continuous differentiability and positive definiteness, so the value function Satisfies the following relationship Among them, f(x,u i (x)) is the dynamic relationship corresponding to the i-th moment; there exists a positive constant σ such that Selecting the value function is the Lyapunov function, and its derivative gives Therefore, the equilibrium point of the closed-loop system is asymptotically stable, that is, the control strategy u (i+1) It is feasible.

Images