A hypersonic vehicle optimization control method based on fast strategy iteration

Abstract

The present invention relates to a hypersonic aircraft optimization control method based on fast strategy iteration, belonging to the field of hypersonic aircraft control technology. First, a hypersonic aircraft longitudinal dynamics model is established; second, the output expected value is substituted into the longitudinal dynamics model, the state expected value and the input expected value corresponding to the output expected value are solved, and the output tracking problem is converted into a state stabilization problem by using coordinate transformation; third, based on feedback linearization theory and sliding mode control method, the initial allowable control law of the hypersonic aircraft control system is designed; finally, an iterative optimization control law using neural network approximation is designed to complete the hypersonic aircraft optimization control method based on fast strategy iteration. The present invention realizes the longitudinal control of the hypersonic aircraft cruise stage, has the characteristics of simple implementation form and optimized control performance, and is suitable for hypersonic aircraft systems that require fast response and have strict control performance requirements.

Description

Technical Field

[0001] The present invention belongs to the technical field of hypersonic aircraft control, and specifically relates to a hypersonic aircraft optimization control method based on rapid strategy iteration, which solves the problems of complex implementation form and high control cost in hypersonic aircraft control based on strategy iteration, and is suitable for hypersonic aircraft systems that require rapid response and have strict control performance requirements. Background Art

[0002] For hypersonic aircraft, the main task of the tracking control system is to control the hypersonic aircraft to the desired speed and altitude and achieve stable level flight. Traditional control methods represented by sliding mode control, backstepping control, adaptive control, dynamic inverse control, linear variable parameter control, etc. are relatively simple in design and are relatively mature. Driven by a new round of technological changes, the performance indicators of hypersonic aircraft control systems need to be optimized, but the above methods do not consider the optimization of performance indicators enough. Optimization control theory can be applied to solve such problems, ensuring that the pre-defined performance function reaches (approximately) the minimum while ensuring the stability of the control system and achieving the control target. For the optimization control problem of discrete-time linear systems using quadratic cost functions, the optimal control can be obtained by solving the discrete Riccati equation; however, for the optimization control problem of general nonlinear systems, it is necessary to solve the Hamilton-Jacobi-Bellman equation (HJB equation) which combines functional and partial differential equations to obtain the optimal solution, but the analytical solution is usually difficult to obtain, especially when the state space dimension is very high, the numerical calculation is very difficult, and it will face the "curse of dimensionality". Therefore, in order to overcome these difficulties, a reinforcement learning method - policy iteration was proposed.

[0003] Many scholars have conducted research and proposed several control methods for hypersonic flight vehicle control based on policy iteration. The paper (Qi Q and Bu X. Adaptive dynamic programing design for the neural control of hypersonic flight vehicles, Journal of the FranklinInstitute, 2021, 358(16): 8169-8192) designed a policy iteration controller based on backstepping method, which used the backstepping method to estimate the unknown function through the neural network, reduced the computational load through the minimum learning parameter scheme, and adopted a first-order filter to avoid the "term explosion" problem. The paper (Greene M, Deptula P, Bialy B, et al. Model-based approximate optimal feedback control of a hypersonic vehicle, AIAASciTech 2022 Forum, 2022) considered the changes in aerodynamic thermoelastic parameters, switched the nominal dynamic model discretely over time to better reflect the changes caused by the parameters, designed the actor-evaluator update law for the policy iteration method based on Lyapunov analysis, and proved the uniformly bounded convergence of the switching dynamic system. The literature (Bao C, Wang P, HeR, et al. Observer-based optimal control method combination with event-triggered strategy for hypersonic morphing vehicle, Aerospace Science and Technology, 2023, 136: 108219) established an attitude-deformation coupling dynamics model, and realized the policy iteration of the approximate optimal feedback control strategy through offline training and exploration, and finally formed a hypersonic aircraft deformation attitude coupling control method based on event-triggered policy iteration and finite-time fuzzy disturbance observer. In Chinese patent application CN202211227304.2, policy iteration is combined with longitudinal guidance and lateral guidance to perform intelligent agent training, and the tangential reaction coefficient and tangential direction coefficient in the lateral guidance method are optimized online, which improves the avoidance performance and adaptability of the aircraft in different flight environments.Chinese patent application CN202310302104.7 introduces a zero-sum game performance indicator, which forms a zero-sum game between the performance of a hypersonic vehicle under state safety constraints and the ability of the hypersonic vehicle attitude dynamics model to compensate for uncertainty, and optimizes the compensation fault-tolerant safety controller through strategy iteration. However, most of the above control methods do not consider the computational cost, and the constructed control law structure is relatively complex, which cannot achieve real-time calculation and control of actual fast-changing objects. For the optimization control problem, the commonly used method is to construct an action network, a model network, and an evaluation network respectively to estimate the controller, system state, and cost function, but it usually requires the collection of a large amount of data and takes a long time for iterative calculation, which is usually not feasible for controlled objects with fast-changing states and limited resources such as hypersonic aircraft.

[0004] In summary, the existing strategy iteration methods lack feasible solutions for fast solutions to the problem of optimal control, and there is an urgent need to develop a fast solution method for the optimal control law of hypersonic aircraft. Summary of the invention

[0005] Aiming at the problems of rapid maneuvering of targets, high requirements for dynamic performance and limited computing resources in the optimization control of hypersonic aircraft, in order to overcome the deficiencies of the prior art, the present invention provides a hypersonic aircraft optimization control method based on fast strategy iteration, designs an initial admissible control law based on feedback linearization theory and a sliding mode control method, solves an overdetermined linear equation group in the least squares sense to obtain a neural network weight that approximately satisfies the Hamilton-Jacobi-Bellman equation (HJB equation for short), designs an iterative optimization control law using a neural network approximation based on fast strategy iteration, realizes a rapid solution of the optimization control law, improves the rapid response performance of the system, and enables the hypersonic aircraft to achieve optimization control using limited computing resources.

[0006] In order to achieve the above object, the present invention adopts the following technical scheme:

[0007] A hypersonic aircraft optimization control method based on rapid strategy iteration comprises the following steps:

[0008] The first step is to establish the longitudinal dynamics model of the hypersonic vehicle;

[0009] The second step is to substitute the output expected value into the longitudinal dynamics model and solve for the state expected value and input expected value corresponding to the output expected value;

[0010] The third step is to transform the output tracking problem into a state stabilization problem by using coordinate transformation, and obtain the deviation system for fast policy iteration optimization control.

[0011] The fourth step is to design the initial admissible control law of the hypersonic vehicle control system based on feedback linearization theory and sliding mode control method;

[0012] The fifth step is to solve the overdetermined linear equations in the least squares sense based on fast strategy iteration to obtain the neural network weights that approximately satisfy the Hamilton-Jacobi-Bellman equations, and then calculate the iterative optimization control law based on the neural network weights.

[0013] The beneficial effects of the present invention compared with the prior art are:

[0014] The invention relates to a fast solution method for an optimization control law. Aiming at the shortcomings of existing methods that require the use of a complex neural network structure for iterative solution, the method does not consider the calculation cost of an actual fast-changing object, and lacks real-time calculation and control capabilities, a least squares solution method for an overdetermined linear equation group of neural network weights is designed, and a fast solution of the neural network weights that approximately satisfy the HJB equation is achieved, so that the iterative optimization control law can be quickly calculated, and the optimization control law for a controlled object with fast-changing states and limited resources such as a hypersonic aircraft is quickly solved. The method has the characteristics of simple implementation form and optimized control performance, and is suitable for a hypersonic aircraft system that requires a fast response and has stringent control performance requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 The present invention is a design flow chart of a hypersonic aircraft optimization control method based on rapid strategy iteration. DETAILED DESCRIPTION

[0016] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0017] like Figure 1 As shown, the present invention discloses a hypersonic aircraft optimization control method based on fast strategy iteration, comprising the following steps:

[0018] The first step is to establish a longitudinal dynamics model of a hypersonic vehicle;

[0019] The second step is to calculate the state expected value and the input expected value under the output expected value;

[0020] The third step is to convert it into a state stabilization optimization control problem through equilibrium point solution and coordinate transformation;

[0021] The fourth step is to design the initial admissible control law based on feedback linearization theory and sliding mode control method;

[0022] The fifth step is to obtain the neural network weights based on the fast policy iteration method and construct the iterative optimization control law.

[0023] The present invention can meet the computing resource constraints and real-time response requirements, improve the rapid computing capability of the control system, has the characteristics of simple implementation and optimized control performance, and is suitable for hypersonic aircraft systems with stringent requirements on control performance.

[0024] Specifically, in the first step, the longitudinal dynamics model of the hypersonic vehicle is established as follows:

[0025] S 1 : { V ˙ = T cos α − D m − G sin c ( R e + h ) 2 c ˙ = L + T sin α mV − [ G − V 2 ( R e + h ) ] c o s c V ( R e + h ) 2 h ˙ = V sin c α ˙ = Q − c ˙ Q ˙ = M yyy I yyy β ¨ =− 2 live n β ˙ − oh n 2 β + oh n 2 β c ,

[0026] in, They are speed, track inclination, altitude, angle of attack, pitch rate, throttle opening and throttle opening command, They are First derivative with respect to time; is the mass, the value is 136818kg, is the gravitational constant, which is 3.979×10 14 m 3 / s 2 , is the moment of inertia, and its value is 9.49×10 6 kg·m 2 , is the damping coefficient, which is 0.7. is the natural frequency, with a value of 1.2, is the radius of the earth, which is 6371387m; They are thrust, aerodynamic drag, aerodynamic lift, and pitching moment. The specific expression of Indicates, respectively:

[0027] S 2 : { T = 1 2 r V 2 S C T = 1 2 r V 2 S ⋅ 0 . 0 2 5 7 6 β D = 1 2 r V 2 S C D = 1 2 r V 2 S ( 0 . 6 4 5 0 α 2 + 0 . 0 0 4 3 3 8 α + 0 . 0 0 3 7 7 2 ) L = 1 2 r V 2 S C L = 1 2 r V 2 S ⋅ 0 . 6 2 0 3 α M y y = 1 2 r V 2 S c ¯ [ C M ( d e ) + C M ( α ) + C M ( Q ) ] = 1 2 r V 2 S c ¯ [ c e ( d e − α ) + 0 . 0 3 5 α 2 + 0 . 0 3 7 α + c ¯ Q 2 V ( − 6 . 8 α 2 + 0 . 3 0 α − 0 . 2 3 ) ] ,

[0028] in, is the elevator deflection angle; is the atmospheric density, which is 0.012537 kg / m 3 , is the reference area, and its value is 334.73m 2 , is the average aerodynamic chord length, which is 24.4m. is the moment coefficient, and its value is 0.0292; , , They are thrust coefficient, aerodynamic drag coefficient, and aerodynamic lift coefficient respectively; Respectively about The pitching moment coefficient.

[0029] Specifically, the second step includes: defining the state x = [ V , c , h , α , Q , β , β ˙ ] T , the superscript "T" indicates the transpose of the matrix; define the input u = [ β c , d e ] T , output y = [ V , h ] T ; Accordingly, define the expected output value y r = [ V r , h r ] T The expected value of the state x r = [ V r , c r , h r , α r , Q r , β r , β ˙ r ] T , enter expected value u r = [ β c , r , d e , r ] T ,in They are The expected value of output y r = [ 4620 . 77 m / s , 34138 m] T Next, order All are zero, combined Solvable :

[0030] ,

[0031] and about The equation :

[0032] .

[0033] Solution , You can get the expected output value The expected value of the system state under and the expected value of the input for:

[0034] x r = [ 4620 . 77 m / s , 0 , 34138 m,0.0306rad,0,0.1751,0 ] T ,

[0035] u r = [ 0 . 1751 , − 0 . 00908 rad ] T .

[0036] Specifically, the third step includes: performing coordinate transformation , , ,in They are state deviation, input deviation, and output deviation, and record The components of e = [ e V , e c , e h , e α , e Q , e β , e β ˙ ] T ,in, They are The deviation from the corresponding reference value is obtained to obtain the transformed deviation system for:

[0037] S 5 :  { e ˙ = f ¯ ( e ) + g ¯ ( e ) v = [ T cos( α r + e α ) − D m − G sin( c r + e c ) ( R e + h r + e h ) 2 L + T sin( α r + e α ) m ( V r + e V ) − [ G − ( V r + e V ) 2 ( R e + h r + e h ) ] c o s ( c r + e c ) ( V r + e V ) ( R e + h r + e h ) 2 ( V r + e V )sin( c r + e c ) Q r + e Q − c ˙ r ( V r + e V ) 2 S c ¯ [ c e ( d e , r − α r − e α ) + C M ( α r + e α ) + C M ( Q r + e Q ) ] 2 I yyy β ˙ r + e β ˙ − 2 g oh n ( β ˙ r + e β ˙ ) − oh n 2 ( β r + e β ) + oh n 2 β c , r ] + [ 0 0 0 0 r ( V r + e V ) 2 2 S c ¯ I y y − 1 c e 0 0 0 0 0 0 0 0 oh n 2 ] T v ,

[0038] in, for The first derivative with respect to time, , is a smooth function, represents the set of all 7-dimensional real column vectors, represents the set of all 7-row and 2-column real matrices.

[0039] Discretization Afterwards :

[0040] ,

[0041] in, is the sampling time, is the sampling period, the value is 0.1s, They are Moment , , , is a smooth function.

[0042] Specifically, the fourth step includes: adopting Respectively The first, second, and third derivatives with respect to time, Respectively The first, second, third, and fourth order derivatives with respect to time. The sliding mode variable for:

[0043] ,

[0044] in," ” is the differential operator, It is a design parameter and its value is 0.2.

[0045] Select the Lyapunov function for:

[0046] ,

[0047] Derivation :

[0048] .

[0049] in Respectively The derivative with respect to time.

[0050] Design control law to obtain :

[0051] ,

[0052] in, express The third derivative with respect to time, express Fourth derivative with respect to time; symbolic function Defined as , It is a design parameter and its value is 1.

[0053] Substitution have:

[0054] ,

[0055] At this time, the inequality Always holds, and:

[0056] ,

[0057] in, Represents the minimum function.

[0058] according to , The required form , using the sliding mode control method to design the initial admissible control law for:

[0059] v s = [ g V g h ] − 1 [ V ⃛ − f V h ⃜ − f h ] =− ϒ − 1 ( C + K sgn( s )) = ϒ − 1 [ − f V − 2 l V e ¨ V − l V 2 e ˙ V − k V s g n ( s e V ) − f h − 3 l h e ⃛ h − 3 l h 2 e ¨ h − l h 3 e ˙ h − k h s g n ( s e h ) ] ,

[0060] in, , ,in They are The control irrelevant items in represents the set of real numbers, They are Control related items in ϒ = [ g V T , g h T ] T , , C = [ f V + 2 l V e ¨ V + l V 2 e ˙ V , f h + 3 l h e ⃛ h + 3 l h 2 e ¨ h + l h 3 e ˙ h ] T , s = [ s e V , s e h ] T , sgn( s ) = [sgn( s e V ),sgn( s e h )] T ,in, is the diagonalization function; The specific expression is:

[0061] { f V ( e ) = w ˙ T ∇ w [ ∇ w T f 1 ( w )] w ˙ + [ ∇ w T f 1 ( w )] w ¨ 0 g V ( e ) = [ ∇ w T f 1 ( w )] i T f h ( e ) = sin( c r + e c ) w ˙ T ∇ w [ ∇ w T f 1 ( w )] w ˙ + sin( c r + e c )[ ∇ w T f 1 ( w )] w ¨ 0 + ( V r + e V ) w ˙ T ∇ w [ ∇ w T f 2 ( w )] w ˙ + ( V r + e V )[ ∇ w T f 2 ( w )] w ¨ 0 + 3 V ¨ c ˙ cos( c r + e c ) − 3 V ˙ c ˙ 2 sin( c r + e c ) + 3 V ˙ c ¨ cos( c r + e c ) − 3 ( V r + e V ) c ˙ c ¨ sin( c r + e c ) − ( V r + e V ) c ˙ 3 cos( c r + e c ) g h ( e ) = sin( c r + e c )[ ∇ w T f 1 ( w )] i T + ( V r + e V )[ ∇ w T f 2 ( w )] i T ,

[0062] Among them, the independent state w = [ e V , e c , e h , e α , e β ] T , for Derivative with respect to time; Represents the gradient ; w ¨ 0 = [ V ¨ , c ¨ , h ¨ , α ¨ 0 , β ¨ 0 ] T ,in They are The second derivative with respect to time, and:

[0063] ,

[0064] f 2 ( w ) = L + T s i n ( α r + e α ) m ( V r + e V ) − [ G − ( V r + e V ) 2 ( R e + h r + e h ) ] c o s ( c r + e c ) ( V r + e V ) ( R e + h r + e h ) 2 ,

[0065] i = [ 0 0 0 0 oh n 2 0 0 0 1 2 r ( V r + e V ) 2 S c ¯ I y y − 1 c e 0 ] ,

[0066] α ¨ 0 = r ( V r + e V ) 2 S c ¯ 2 I yyy [ C M ( α r + e α ) + c e ( d e , r − α r − e α ) + C M ( Q r + e Q ) ] − c ¨ ,

[0067] .

[0068] Specifically, the fifth step includes: the utility function is defined as , where the positive definite symmetric matrix , positive definite symmetric matrix , the cost function is defined as , is the intermediate variable. Select the basis function ϕ ( e k ) = [ e k T , v e c T ( e k e k T ) ] T ∈ ℝ 56 ,in is the column expansion operator, Represents the set of all 56-dimensional real column vectors and defines the weights W ^ = [ W ^ 1 , W ^ 2 , . . . , W ^ 5 6 ] T ∈ ℝ 5 6 And construct ,in Express 500 samples are collected to obtain ,in, is the sample order, The first state deviations, and obtain the weights for:

[0069] W ^ = [ ( D ϕ S ) T D ϕ S ] − 1 ( D ϕ S ) T U S ,

[0070] Among them, the basis function increment vector D ϕ S = [ D ϕ ( e k , 1 ) , D ϕ ( e k , 2 ) , . . . , D ϕ ( e k , 5 0 0 ) ] T ∈ ℝ 5 0 0 × 5 6 , basis function increment Defined as , the utility function vector U S = [ U ( e k , 1 , v s ( e k , 1 ) ) , U ( e k , 2 , v s ( e k , 2 ) ) , . . . , U ( e k , 5 0 0 , v s ( e k , 5 0 0 ) ) ] T ∈ ℝ 500 , They represent the set of all 500-row and 56-column real matrices, the set of 56-dimensional real column vectors, and the set of 500-dimensional real column vectors. is the smooth function mentioned above, is the initial admissible control law mentioned above.

[0071] Solving by minimal necessary conditions , we can get the iterative optimization control law (denoted as )(where the subscript p represents policy-iteration) satisfies :

[0072] .

[0073] From the definition of utility function, we know that :

[0074] .

[0075] When the sampling rate is high enough, it can be approximated that , so there is :

[0076] .

[0077] because , so we get :

[0078] .

[0079] Depend on Knowable :

[0080] .

[0081] comprehensive to It can be solved:

[0082] .

[0083] in, is the positive definite symmetric matrix mentioned above, express right The partial derivative of .

[0084] The method of the present invention is adopted to carry out longitudinal control of a hypersonic aircraft, and an iterative optimization control law based on fast strategy iteration is designed. The tracking control task can be completed under the specified altitude and speed, and the given cost function can be reduced by 37%. At the same time, the calculation cost is greatly reduced, and the optimized control of the hypersonic aircraft is realized. The problems of complex implementation form and high calculation cost of the optimized control are solved, and the requirements of fast dynamic response and optimized performance indicators are met.

[0085] The contents not described in detail in the specification of the present invention belong to the prior art known to the professional and technical personnel in this field.

Claims

1. A hypersonic vehicle optimization control method based on fast strategy iteration, characterized in that: The following steps are involved: The first step is to establish the longitudinal dynamics model of the hypersonic vehicle; The second step is to substitute the output expected value into the longitudinal dynamics model and solve for the state expected value and input expected value corresponding to the output expected value; The third step is to transform the output tracking problem into a state stabilization problem by using coordinate transformation, and obtain the deviation system for fast policy iteration optimization control. The fourth step is to design the initial admissible control law of the hypersonic vehicle control system based on feedback linearization theory and sliding mode control method, including: The input deviation to be designed is , the initial admissible control law is designed by using the sliding mode control method for: ， Wherein, the subscript s represents the sliding mode; , ,in They are The control irrelevant items in represents the set of real numbers, They are Control related items in They are The first, second, and third order derivatives with respect to time, They are First, second, third, and fourth order derivatives with respect to time; are design parameters; symbol function Defined as ; The fifth step is to solve the overdetermined linear equations in the least squares sense based on fast policy iteration to obtain the neural network weights that satisfy the Hamilton-Jacobi-Bellman equation, and then calculate the iterative optimization control law based on the neural network weights; The utility function is defined as , where the positive definite symmetric matrix , positive definite symmetric matrix , and Respectively represent the set of all 7th-order and 2nd-order real square matrices, and the cost function is defined as , is the intermediate variable; select the basis function , Represents all A collection of dimensional real column vectors, is the dimension of the basis function, and defines the weight And construct ,in Express The estimate of ,in is the sample order, is the number of samples, is the first state deviations, and obtain the weights for: ， Among them, the basis function increment vector , Represents all OK A collection of column-real matrices, basis function increments Defined as , is a smooth function, is the initial admissible control law, the utility function vector ; Then the iterative optimization control law is solved : ， Among them, the subscript p represents the policy iteration, is a positive definite symmetric matrix, express right The partial derivative of .

2. The hypersonic vehicle optimization control method based on fast strategy iteration according to claim 1, characterized in that: In the second step, define the system state ,enter and output ,in They are speed, track inclination, altitude, angle of attack, pitch rate, throttle opening, for The first derivative with respect to time, the control signal are the throttle opening command and the elevator deflection angle respectively. The superscript "T" indicates the transpose of the matrix. Accordingly, the output expected value is defined as The expected value of the state , enter expected value ,in They are The expected value of Next, we can get the angle of attack Expected value The equation, such as As shown, solving Get the expected output value The expected value of the system state under and the expected value of the input u ; ， in, are the fitting coefficients of aerodynamic forces, They are mass, gravitational constant, atmospheric density, reference area, and radius of the earth.

3. The hypersonic vehicle optimization control method based on fast strategy iteration according to claim 2 is characterized in that: In the third step, coordinate transformation is performed , , ,in They are state deviation, input deviation, and output deviation, and record The components of ,in They are The deviation from the corresponding expected value gives the transformed deviation system for: ， in, for The first derivative with respect to time, , is a smooth function, represents the set of all 7-dimensional real column vectors, represents the set of all 7-row and 2-column real matrices, They are thrust, aerodynamic drag, and aerodynamic lift, respectively. are the average aerodynamic chord length, torque coefficient, moment of inertia, damping coefficient, natural frequency, earth radius, gravitational constant, Respectively about The pitching moment coefficient of Discretization Then we get: ， in, is the sampling time, is the sampling period, They are Moment , , , is a smooth function.

Images